Automate Airflow DAG development with TaskFlow API and GCP integrations.
npx skills add https://github.com/ilorozco11/agent-skill --skill airflow-dag
Please help me install this Agent Skill. Skill: airflow-dag Source: https://github.com/ilorozco11/agent-skill/tree/main/.github/skills/airflow-dag Command: npx skills add https://github.com/ilorozco11/agent-skill --skill airflow-dag
This Skill provides ready-to-use Airflow DAG templates and best practices for building, testing, and deploying robust data pipelines using the TaskFlow API and Google Cloud operators. It also covers cross-DAG dependencies, deferrable operators, error handling, retries, and dbt integration to streamline orchestration in modern data environments.
Launch a sample Airflow DAG template to validate your environment.
High-intent search queries and answers about installing and using this skill.
Building Airflow DAGs with the TaskFlow API is streamlined using ready-to-use templates that create modular and testable pipelines. These patterns support clear task orchestration and integrate cloud operators for end-to-end data processing.
Handling cross-DAG dependencies in Apache Airflow is best managed using ExternalTaskSensor patterns and robust error handling. This approach ensures reliable orchestration across multiple data pipelines without failing silently.
Integrating dbt workflows with BigQuery and GCS in an Airflow pipeline is achieved through specialized cloud operators. You can extract data from BigQuery, process it with dbt, and load the results back to BigQuery seamlessly.
Deferrable operators in Airflow can be used to reduce resource consumption by releasing worker slots while waiting for external conditions. This Skill provides patterns to implement these operators effectively within your data pipelines.
OpenLineage integration is supported by these Airflow DAG templates to provide data observability and track pipeline metadata. This feature works alongside dynamic DAG generation and error handling in GCP environments.
Adding retries and error handling to a daily recommender data pipeline in Airflow ensures robust execution even when temporary failures occur. The templates provide built-in mechanisms to handle errors during BigQuery and dbt operations.
astronomerConvert dbt Core projects into Airflow DAGs or TaskGroups using Astronomer Cosmos.
erichowensAutomate ETL/ELT pipeline design, validation, and operation across medallion architectures.
AbsolutelySkilledEnforces data quality with validation, lineage tracking, and contracts.
tan-yong-shengAutomate cloud infrastructure deployment across Cloudflare, Docker, Google Cloud, and Kubernetes.
