airflow-workflows

Name: airflow-workflows
Availability: InStock
Author: timequity

Automate ETL pipeline orchestration with Apache Airflow DAGs.

6|Updated Dec 7, 2025

Search Tags:#workflow #etl #airflow #data-pipelines #dags

Ready to run

npx skills add https://github.com/timequity/plugins --skill airflow-workflows

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: airflow-workflows
Source: https://github.com/timequity/plugins/tree/main/craft-coder/data/airflow-workflows
Command: npx skills add https://github.com/timequity/plugins --skill airflow-workflows

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides guidance for designing robust Airflow DAGs, selecting operators, and scheduling patterns to orchestrate data workflows reliably.

Core Features & Use Cases

DAG Structure: Best practices for task grouping, dependencies, and retries.
Operators & Tasks: Common Python, Bash, SQL operators and dynamic DAGs.
Scheduling & Reliability: Idempotent tasks, templating, and sensors.

Quick Start

Create a daily_etl DAG using the recommended pattern, and run Airflow locally or in your environment to validate orchestration.

Frequently Asked Questions about airflow-workflows

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I design a robust ETL pipeline with Airflow DAGs?▼

Design ETL pipelines by structuring DAGs with clear task dependencies, selecting appropriate operators (PythonOperator, BashOperator, SQLExecuteQueryOperator), and implementing idempotent tasks with templating using {{ ds }}. Apply retry and alert configurations to ensure reliability across daily, batch, and streaming workflows in cloud or on-prem environments.

What's the best way to orchestrate data workflows across multiple environments?▼

Orchestrate workflows using Apache Airflow DAGs with modular structure, proper operator coverage, and scheduling patterns that support both cloud and on-premises deployments. Configure task dependencies, sensors, and XCom usage minimally to maintain clarity and reliability across different infrastructure setups.

How do I schedule and automate batch and streaming data pipelines?▼

Automate data pipelines using Airflow's scheduling capabilities combined with DAG design patterns that enforce task dependencies and operator selection. Implement templating with {{ ds }} and idempotent task logic to ensure reliable daily, batch, and streaming workflow execution.

Can I use Airflow with Snowflake and dbt for data transformations?▼

Yes. Airflow supports Snowflake integration through S3ToSnowflakeOperator and dbt orchestration via DbtCloudRunJobOperator, enabling you to build end-to-end ETL pipelines that combine data loading, transformation, and scheduling within a single DAG structure.

What operator should I use for different task types in Airflow?▼

Select operators based on task requirements: PythonOperator for Python scripts, BashOperator for shell commands, SQLExecuteQueryOperator for database queries, S3ToSnowflakeOperator for cloud data loading, and DbtCloudRunJobOperator for dbt transformations. Proper operator selection ensures efficient, maintainable DAG design.

Why does minimal XCom usage matter in Airflow DAG design?▼

Minimizing XCom usage prevents performance degradation and state management complexity in Airflow DAGs. Following best practices for task grouping, dependencies, and data passing keeps workflows efficient and maintainable at scale.