airflow-workflow

What problem does it solve?

Data engineering teams face challenges managing Apache Airflow scheduled SQL jobs, including troubleshooting failed runs without structured guidance, safely updating job configurations without disrupting active workflows, and adhering to connection and naming conventions to avoid costly misconfigurations and downtime.

Core Features & Use Cases

• Full Job Lifecycle Management: Safely create, update, pause, resume, and delete Airflow scheduled SQL jobs with built-in guardrails to prevent accidental data loss or unplanned downtime.
• Structured Troubleshooting Workflow: Step-by-step guidance to diagnose failed job runs, identify common failure causes (SQL syntax errors, connection failures, timeouts, permission issues), and implement validated fixes.
• Best Practice Enforcement: Built-in naming conventions for jobs and SQL files, plus a reference of common cron expressions to ensure consistent, maintainable workflow scheduling.
• Real-World Use Case: A data engineer needs to resolve a failing daily sales summary job that is throwing a connection error; this skill guides them to verify the correct conn_id in Airflow Connections, update the job configuration, and run a manual test to confirm the fix works.

Quick Start

Use the airflow-workflow skill to troubleshoot and resolve the failed hourly_order_count Airflow job that is throwing a permission denied error.