skills.rest
Jhabbig
JhabbigCOMMUNITY
Explore all skills by this author →
GitHub

airflow-dag-patterns

Build Apache Airflow DAGs with task dependencies, sensors, and retries.

Updated Apr 5, 2026
Search Tags:#testing#sensors#orchestration#etl#airflow#operators#dag
One-click install
npx skills add https://github.com/Jhabbig/Habbig --skill airflow-dag-patterns-jhabbig
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: airflow-dag-patterns
Source: https://github.com/Jhabbig/Habbig/tree/main/.claude/plugins/wshobson/data-engineering/skills/airflow-dag-patterns
Command: npx skills add https://github.com/Jhabbig/Habbig --skill airflow-dag-patterns-jhabbig

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you design, debug, and productionize Airflow workflows without piecing together patterns from scattered examples. It reduces common orchestration mistakes such as brittle dependencies, unreliable retries, and poorly tested DAGs.

Core Features & Use Cases

  • DAG design patterns: Build clear task graphs with linear, fan-out, fan-in, and branching dependencies.
  • Operators and sensors: Choose the right Python, branching, external-task, and sensor patterns for real pipelines.
  • Testing and reliability: Validate DAG imports, structure, cycles, and task behavior before deployment.
  • Deployment readiness: Apply retry, timeout, trigger-rule, and callback strategies for production scheduling.
  • Use case: Use this Skill when creating an ETL pipeline that waits on upstream data, branches by quality checks, and alerts on failures.

Quick Start

Ask for an Apache Airflow DAG pattern for your pipeline, including task dependencies, sensors, retries, and tests.

Frequently Asked Questions about airflow-dag-patterns

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a production-ready Airflow DAG with sensors and branching logic?▼

To build a production-ready Airflow DAG, apply patterns for task dependencies, external-task sensors, and branching operators to create clear, reliable task graphs for data pipelines.

What is the best way to test Airflow DAGs for cycles and import errors before deployment?▼

The best way to test Airflow DAGs is to validate DAG imports, structure, cycles, and task behavior locally before deployment, ensuring atomic task behavior and idempotency for reliable orchestration.

How do I handle Airflow retries and trigger rules for batch scheduling?▼

To handle Airflow retries and trigger rules for batch scheduling, apply deployment readiness strategies including timeouts, retry configurations, and callback alerts to ensure dependable workflow orchestration.

When do I need to use sensors in an Airflow ETL pipeline?▼

You need to use sensors in an Airflow ETL pipeline when your workflow must wait on upstream data availability or external task completion before proceeding with incremental processing.

Can I use branching operators to skip failed data quality checks in Airflow?▼

Yes, you can use branching operators in Airflow to route pipeline execution based on data quality checks, enabling dynamic task graph generation and conditional downstream processing.

Related Skills

Explore More Software Engineering Skills→
wshobsonwshobson
38.6k

airflow-dag-patterns

Provide production-ready Apache Airflow DAG design patterns with operators, sensors, testing, and deployment strategies.

Community
Advanced
binfen1binfen1

canary

Monitor live apps for console errors, performance regressions, and page failures.

Community
Advanced
haxlyshaxlys

airflow-dag-patterns

Create production-ready Airflow DAGs for ETL pipelines and batch jobs.

Community
Advanced
as4584as4584
2

airflow-dag-patterns

Generate Apache Airflow DAG patterns with task dependencies and testing examples.

Community
Advanced