airflow-expert

To write an Airflow DAG with the TaskFlow API, use Python decorators to define tasks and pass data directly, abstracting explicit XCom management. This Skill guides you through structuring DAGs and applying best practices for production.

Optimizing Airflow DAG parsing and execution requires avoiding anti-patterns and applying performance tuning. This Skill provides expert guidance on scheduling, operator selection, and resolving bottlenecks for Airflow 2.7+ and 3.x.

Choosing the right Airflow operators and sensors depends on your specific data pipeline tasks. This Skill recommends appropriate operators and sensors for data extraction and loading, helping you structure reliable daily data processing workflows.

Troubleshooting common Apache Airflow DAG errors involves diagnosing issues with scheduling, XCom management, and sensor usage. This Skill assists in diagnosing and resolving these production deployment failures efficiently.

Yes, this Airflow expert guidance supports version 3.x, along with Airflow 2.7+. It covers modern DAG development, TaskFlow API usage, and performance tuning tailored for these specific production environments.

You should avoid using Airflow sensors improperly when they block worker slots for extended periods, a common anti-pattern in workflow orchestration. This Skill helps identify such limitations and recommends best practices for sensor usage.