airflow-dag-patterns

What problem does it solve?

This skill provides a standardized set of best practices and patterns for designing Airflow DAGs to improve maintainability, readability, and reliability.

Core Features & Use Cases

• Standardized DAG templates and naming conventions to ensure consistency across teams.
• Clear task flow patterns and modular design to simplify complex pipelines.
• Robust error handling, retries, and monitoring guidance to reduce failures and manual interventions.
• Use case: quickly onboard a new data source by scaffolding a DAG that follows the same patterns.

Quick Start

Integrate the airflow-dag-patterns into your project by adopting the provided DAG patterns and task-naming conventions. Start with dags/multi_sport_betting_workflow.py as your template and adapt it for your data sources and tasks. Validate the DAGs in Airflow by listing and triggering them in your environment.