Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POCs, and other uses in Databricks environments including in Delta Live Tables pipelines

The Metadata Driven framework for Databricks Lakeflow Declarative Pipelines (formerly Delta Live Tables). Metadata framework that generates production ready Pyspark code for Lakeflow Declarative Pipelines

A lightweight helper utility which allows developers to do interactive pipeline development by having a unified source code for both DLT run and Non-DLT interactive notebook run.

Notebooks to learn Databricks Lakehouse Platform

Databricks DLT Apparel Pipeline Project: Learn medallion architecture, streaming, and data engineering with Delta Live Tables. Includes synthetic data, step-by-step guide, and certification prep.

Demonstration of using Files in Repos with Databricks Delta Live Tables

This real-time project integrates flight information from the AviationStack API for DFW Airport and weather data from the National Weather Service API, to provide the latest arrival, departure, and forecast details.

Schema mappings in SQL and PySpark for ELT pipelines to normalize data to OCSF

Deploying Delta Live Tables Pipelines on AWS with Terraform

Azure End To End Data Engineering Project | Azure Data Factory | Azure Databricks | Azure SQL DB | PySpark | Big Data. It is a in depth Data Engineering project using powerful tools like Azure Data Factory, Azure SQL DB, Azure Databricks, Unity Catalog, Delta Live Tables, Spark Streaming, PySpark, Databricks Asset Bundles, GitHub, and more.

Real Estate ELT pipeline using Databricks Asset Bundles on GCP. Ingests, transforms, and analyzes property data via Delta Live Tables. Follows medallion architecture (Bronze/Silver/Gold), modular Python design, CI/CD automation with GitHub Actions, and full Unit and Integration tests coverage.

End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

End-to-end Azure data engineering pipeline for Spotify data using Azure Data Factory, Databricks, PySpark, and Delta Live Tables with a medallion architecture.

Production-ready data pipeline using Azure Databricks Delta Live Tables with Medallion Architecture. Processes airline data via streaming (Event Hubs) and batch (ADLS Gen2) ingestion, enforces data quality constraints, and delivers analytics through Power BI dashboards. Demonstrates DLT, CDC, and Auto Loader.

End-to-end Azure Databricks data engineering project using Autoloader, Delta Live Tables (DLT), and Medallion Architecture.

A concise handbook covering Databricks architecture, Delta Lake, Unity Catalog, and data engineering best practices for building scalable and production-ready data pipelines.

This project implements a modern data engineering pipeline using Databricks, PySpark, DBT, and Delta Live Tables. It follows the Medallion Architecture, supports realtime data ingestion with Autoloader, and models data with fact and dimension tables, including Slowly Changing Dimensions (SCD Type 2), all orchestrated in a scalable cloud environment

Production-grade AWS + Databricks data quality platform. Multi-engine (Great Expectations + Soda + custom PySpark + DLT), DynamoDB results catalog with 3 GSIs, Unity Catalog lineage, SLO-driven burn-rate alerting (Slack + PagerDuty), Grafana dashboards, 15 Terraform modules.

Multi-tenant IoT telemetry Lakehouse on Databricks + Delta Lake. PySpark, Auto Loader, DLT, medallion architecture, Terraform IaC.

Examples of Databricks Delta Live Tables