Udemy - Master Airflow - Orchestrate Snowflake and Databricks

Master Airflow: Orchestrate Snowflake & Databricks

https://WebToolTip.com

Published 7/2026
MP4 | Video: h264, 1920x1080 | Audio: AAC, 44.1 KHz, 2 Ch
Language: English | Duration: 5h 41m | Size: 1.58 GB

Stop firefighting cron chains — build production Airflow 3 DAGs that orchestrate Snowflake, Databricks, and dbt.

What you'll learn
Explain why chained cron jobs silently corrupt downstream data, and what a DAG's explicit completion check fixes
Author production-grade DAGs using both classic operators and the TaskFlow API's `@task`/`@dag` decorators
Design idempotent, backfillable pipelines using Airflow's data-interval scheduling model
Use sensors and deferrable operators to wait on external systems efficiently, without wasting a worker slot
Pass data between tasks correctly with XComs, understanding what should and shouldn't cross that boundary
Scale a single task over an unknown number of inputs with dynamic task mapping (`.expand()`)
Build branching logic, trigger rules, and human-in-the-loop approval steps into a DAG
Manage credentials safely with Connections, Variables, Hooks, and a Secrets backend — never hardcoded
Orchestrate real cross-platform pipelines: Snowflake loads, Databricks jobs, and dbt runs (via Cosmos), from a single DAG
Debug failed DAG runs using logs, the Grid and Gantt views, retries, SLAs, and alerting callbacks
Deploy Airflow to production (Kubernetes, Astronomer, MWAA, or Cloud Composer) with CI/CD gating every DAG change
Migrate a legacy Airflow 2 deployment to Airflow 3, and recognize the performance anti-patterns that break DAGs at scale

Requirements
Basic Python — functions, imports, and reading a script is enough; no advanced Python required
Comfort with the command line for running Docker and Airflow CLI commands
Docker installed on your machine (used to stand up a local Airflow environment)
No prior Airflow experience required — we build the mental model from zero starting in Module 1
Helpful but not required: prior exposure to Snowflake, Databricks, or dbt, since Module 13 orchestrates all three