managed-airflow-dag-troubleshooting

Diagnose failed Airflow DAG runs and task instances in Managed Service for Apache Airflow.

Updated May 11, 2026
One-click install
npx skills add https://github.com/alon3153/upe-social-publisher --skill managed-airflow-dag-troubleshooting-alon3153
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: managed-airflow-dag-troubleshooting
Source: https://github.com/alon3153/upe-social-publisher/tree/main/.agents/skills/managed-airflow-dag-troubleshooting
Command: npx skills add https://github.com/alon3153/upe-social-publisher --skill managed-airflow-dag-troubleshooting-alon3153

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When a DAG run or task instance fails in Managed Service for Apache Airflow (formerly Cloud Composer), finding the root cause requires digging through Cloud Logging, task metadata, and remote DAG source code. This Skill provides a deterministic, evidence-based troubleshooting workflow that pinpoints the exact failure cause instead of offering generic advice. ## Core Features & Use Cases - Log-driven diagnosis: Fetches task and scheduler logs with gcloud logging read, matching error patterns like Python tracebacks, API error codes, and Airflow signals such as AirflowTaskTimeout. - Task and DAG metadata inspection: Retrieves task states, try numbers, and execution details via gcloud composer environments run commands to identify the critical failing task. - Remote code correlation: Downloads DAG source code from the environment's GCS bucket and compares configuration values (table IDs, URIs, disk sizes) against log errors. - Known platform issue detection: Matches failures against six documented failure modes including DAG_RUN_TIMEOUT, DETECTED_ZOMBIE, and WORKER_OUT_OF_POD_STORAGE. - Orchestration Pipeline support: Resolves generated DAGs back to their pipeline YAML definitions and logical action names. - Use Case: A BigQuery task in your nightly DAG fails with a 404 error. The Skill fetches the worker logs, downloads the DAG source, and shows that the dataset ID in the code does not match the actual BigQuery dataset, then suggests the corrected line. ## Quick Start Ask the assistant to troubleshoot why a specific DAG run failed in your Managed Airflow environment, providing the environment name, location, DAG ID, and run ID.

Frequently Asked Questions about managed-airflow-dag-troubleshooting

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I troubleshoot a failed Airflow DAG run in Cloud Composer?▼

Fetch task states with gcloud composer environments run tasks states-for-dag-run to identify the failed task, then read its logs via gcloud logging read filtered by dag_id and severity. Compare error messages against the DAG source code downloaded from the environment bucket.

How to find Airflow task failure logs with gcloud logging?▼

Use gcloud logging read with resource.type="cloud_composer_environment", the environment name, dag_id, and severity>=ERROR. Format the output with timestamp, severity, task_id, and textPayload to see the exact error text for each task instance.

Why did my Airflow task fail with a zombie or heartbeat error?▼

A zombie task occurs when heartbeats are missing for longer than scheduler_zombie_task_threshold (300 seconds by default), usually from worker CPU or memory starvation, OOM kills, or an overloaded metadata database. Check worker metrics and consider scaling worker resources.

Can this Skill fix my DAG code automatically?▼

No, it follows a read-only-first approach and only diagnoses the root cause with log and code evidence. Fixes are applied only when explicitly requested, and it first verifies the workspace is the correct source repository for the environment.

Does it support DAGs generated by Orchestration Pipelines?▼

Yes, it detects pipeline-generated DAGs via the op:orchestration_pipeline tag and DagRun metadata, retrieves the resolved pipeline YAML from the environment bucket, and maps failed tasks back to logical action names.

When should I not use this troubleshooting skill?▼

Do not use it for general Managed Airflow environment performance recommendations or capacity planning. It is scoped specifically to diagnosing failed DAG runs and task instances, not overall environment tuning.