skills.rest
google
googleOFFICIAL
Explore all skills by this author →
Website𝕏@GoogleOSSGitHub

managed-airflow-dag-troubleshooting

Diagnose failed Airflow DAG runs and task instances in Managed Service for Apache Airflow.

19.1k|1.5k|Updated Mar 31, 2026
Search Tags:#gcloud#Airflow#Google Cloud#Troubleshooting#DAG#Log Analysis#Cloud Composer
One-click install
npx skills add https://github.com/google/skills --skill managed-airflow-dag-troubleshooting
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: managed-airflow-dag-troubleshooting
Source: https://github.com/google/skills/tree/main/skills/cloud/managed-airflow-dag-troubleshooting
Command: npx skills add https://github.com/google/skills --skill managed-airflow-dag-troubleshooting

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

When a DAG run or task instance fails in Managed Service for Apache Airflow (formerly Cloud Composer), finding the root cause requires digging through scattered logs, task metadata, and remote DAG source code. This Skill provides a deterministic troubleshooting workflow that pinpoints the exact failing task, the verbatim error evidence, and an actionable fix.

Core Features & Use Cases

  • Deterministic root cause analysis: Fetches task logs via gcloud logging, task states via gcloud composer, and remote DAG source code from the environment bucket to ground every diagnosis in evidence.
  • Known platform issue detection: Matches failures against documented failure modes such as DAG run timeouts, zombie tasks, worker out-of-storage, and BigQuery job failures using targeted Cloud Logging queries.
  • Orchestration Pipeline DAG support: Detects DAGs generated by Orchestration Pipelines, retrieves the resolved pipeline YAML, and maps failed tasks back to logical pipeline actions.
  • Use Case: A data engineer sees a nightly DAG run marked as failed. The Skill identifies the critical failed task instance, shows the exact BigQuery 404 error from the logs, compares the table ID against the DAG source code, and provides the corrected Python snippet.

Quick Start

Troubleshoot why my DAG run failed in the Managed Airflow environment named my-env in us-central1 and show me the root cause with log evidence.

Frequently Asked Questions about managed-airflow-dag-troubleshooting

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I troubleshoot a failed Airflow DAG run in Cloud Composer?▼

Fetch task instance states with gcloud composer environments run tasks states-for-dag-run to identify the failed task, then read its error logs with gcloud logging read. Compare the error against the remote DAG source code downloaded from the environment bucket to determine the root cause.

How to get Airflow task logs from Google Cloud Composer?▼

Use gcloud logging read with a filter on resource.type cloud_composer_environment, the environment name, dag_id, and severity ERROR. This returns timestamped log entries with textPayload containing tracebacks and API error codes for the failing task.

Why did my Airflow task fail with a zombie or heartbeat error?▼

A zombie task occurs when heartbeats are missing beyond the scheduler_zombie_task_threshold, usually because a worker is overloaded, was OOM-killed, or the metadata database is overloaded. Check worker CPU and memory metrics and consider scaling worker resources.

Can this skill troubleshoot DAGs generated by Orchestration Pipelines?▼

Yes. It detects pipeline-generated DAGs via the op:orchestration_pipeline tag and DagRun metadata, retrieves the resolved pipeline YAML from the environment bucket, and maps the failed Airflow task back to the logical pipeline action name.

When should I not use this Airflow DAG troubleshooting skill?▼

Do not use it for general Managed Airflow environment performance recommendations or tuning advice. It is scoped specifically to diagnosing failed DAG runs and task instances, not overall environment optimization.

Related Skills

Explore More Data & Analytics Skills→
WhaleylawWhaleylaw

cloud-run-deployment-specialist

Deploy containerized applications to Google Cloud Run with gcloud.

Community
Advanced
rory-datarory-data

testing-dags

Trigger Airflow DAG runs, diagnose failures, inspect logs, and apply fixes in iterative cycles.

Community
Advanced
jtucker9jtucker9

cron-scheduler

Generate cron job specifications with overlap prevention and health monitoring.

Community
Advanced
edjan11edjan11

webapp-testing

Automate testing and debugging of local web applications with Playwright.

Community
Intermediate