gcp-managed-airflow-recommendations

Name: gcp-managed-airflow-recommendations
Availability: InStock
Author: SmileAfterBurn

Diagnose Cloud Composer environments and generate evidence-based Airflow tuning recommendations from monitoring metrics.

Updated Aug 26, 2026

Search Tags:#monitoring #gcp #diagnostics #performance tuning #promql #apache airflow #cloud composer

Ready to run

npx skills add https://github.com/SmileAfterBurn/pani-dumka-ai --skill gcp-managed-airflow-recommendations-smileafterburn

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: gcp-managed-airflow-recommendations
Source: https://github.com/SmileAfterBurn/pani-dumka-ai/tree/main/.gemini/skills/gcp-managed-airflow-recommendations
Command: npx skills add https://github.com/SmileAfterBurn/pani-dumka-ai --skill gcp-managed-airflow-recommendations-smileafterburn

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Cloud Composer (Managed Service for Apache Airflow) environments often suffer from workload restarts, CPU/memory saturation, and slow DAG parsing, but diagnosing root causes requires querying scattered metrics and logs. This Skill gathers real telemetry from Cloud Monitoring and turns it into concrete, data-backed tuning recommendations instead of guesswork. ## Core Features & Use Cases - Telemetry Collection: Runs Python scripts that query Cloud Monitoring via PromQL for CPU, memory, disk usage, restart counts, DAG parsing stats, and environment health. - Configuration Inspection: Uses gcloud commands from the reference guide to inspect environment size, scaling parameters, and bucket contents. - Structured Recommendations: Produces categorized advice covering infrastructure scaling, airflow.cfg overrides, bucket hygiene, and production best practices. - Use Case: When a scheduler keeps restarting, run the workload_restarts and workload_memory_usage scripts to confirm an OOM condition, then recommend right-sized memory limits or DAG optimizations rather than blindly upsizing. ## Quick Start Analyze my Cloud Composer environment 'prod-airflow' in project my-gcp-project, region us-central1, over the past 24 hours and give me tuning recommendations.

Frequently Asked Questions about gcp-managed-airflow-recommendations

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I diagnose Cloud Composer performance issues?▼

Run the provided Python scripts with your project ID, location, environment name, and query duration in hours. They fetch CPU, memory, disk, restart, and DAG parsing metrics from Cloud Monitoring via PromQL, giving you data to identify saturation or instability.

How to find why Airflow scheduler keeps restarting?▼

Use the workload_restarts.py script to get restart counts per component, then check workload_memory_usage.py for memory saturation indicating OOM kills. High scheduler restarts often point to memory limits or unoptimized DAGs blocking the event loop.

What metrics does Cloud Composer expose for monitoring?▼

Cloud Composer publishes metrics under composer.googleapis.com including environment health, database health, webserver health, workload CPU/memory/disk usage, restart counts, and DAG processing stats like total parse time and parse error counts.

Does this work with both Cloud Composer 2 and 3?▼

Yes, the scripts query the cloud_composer_environment and cloud_composer_workload monitored resources used by managed Airflow environments. Recommendations should distinguish between generations since their architectures and scaling options differ.

Why is my Airflow DAG parsing slow?▼

Slow parsing usually comes from heavy top-level code in DAG files, such as database connections or API calls outside task definitions. Run dag_parsing_stats.py to measure total parse time and processor timeouts, then move heavy logic into operators.

What permissions are needed to run the monitoring scripts?▼

The scripts authenticate via gcloud application-default credentials and call the Cloud Monitoring PromQL query API. Your account needs permission to read monitoring time series data in the target GCP project.