skills.rest
minyeamer
minyeamerCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

authoring-dags

Author and validate Apache Airflow DAGs using a discovery-plan-implement-validate-test workflow.

3|Updated Jul 22, 2025
Search Tags:#best practices#debugging#airflow#cli validation#taskflow api#etl orchestration#dag authoring
One-click install
npx skills add https://github.com/minyeamer/linkmerce --skill authoring-dags-minyeamer
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: authoring-dags
Source: https://github.com/minyeamer/linkmerce/tree/main/.agents/skills/authoring-dags
Command: npx skills add https://github.com/minyeamer/linkmerce --skill authoring-dags-minyeamer

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

It solves the problem of repeatedly breaking or shipping faulty Apache Airflow DAGs by providing a structured, best-practice workflow for discovering context, implementing pipelines, validating DAG correctness, and running safe tests.

Core Features & Use Cases

  • End-to-end DAG authoring workflow: guides discover → plan → implement → validate → test → iterate to reduce integration mistakes.
  • Airflow CLI-based validation loop: instructs how to use af commands to check connections/variables/providers/version and to debug DAG parse errors and runtime behavior.
  • Best-practice guardrails: reinforces patterns like avoiding top-level code, using TaskFlow API, idempotency, data intervals, and safe retry/validation habits when authoring ETL pipelines.

Quick Start

Create a new Airflow DAG using the authoring-dags skill, then validate it with af dags errors before you trigger any run.

Frequently Asked Questions about authoring-dags

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate Airflow DAGs before triggering a run?▼

Airflow DAG authoring best practices include avoiding top-level code, using the TaskFlow API, ensuring idempotency, and correctly managing data intervals. Following these patterns reduces integration mistakes and prevents runtime errors in ETL pipelines.

What's the best way to structure Airflow DAG authoring to reduce shipping errors?▼

The best way to structure Airflow DAG authoring is using a discover, plan, implement, validate, and test workflow. This repeatable process enforces DAG conventions and uses CLI-driven feedback loops to catch errors early.

How does the TaskFlow API work in Airflow for ETL orchestration?▼

The TaskFlow API in Airflow simplifies ETL orchestration by defining tasks as Python functions, reducing boilerplate code. It is a recommended best-practice pattern for writing clean, maintainable DAGs without explicit operator dependencies.

Why does my Airflow DAG break during parsing?▼

Airflow DAG parsing often breaks due to top-level code execution or missing provider dependencies. Use the `af` CLI commands to debug parse errors, check installed providers, and validate your Airflow version configuration.

Can I use CLI validation to check Airflow connections and variables?▼

Yes, you can use Airflow CLI commands to check connections, variables, providers, and the Airflow version. This CLI-driven validation loop helps debug DAG parse errors and verify runtime behavior safely.

Related Skills

Explore More Software Engineering Skills→
datasurfacedatasurface

DataSurface Setup Walkthrough

Guides step-by-step deployment of DataSurface Yellow on Docker Desktop via Helm and kubectl.

Official
Advanced
astronomerastronomer
419

testing-dags

Trigger Airflow DAG runs, diagnose failures, and inspect task logs.

Official
Advanced
yusufcmgyusufcmg
1

airflow-dag-patterns

Generate production-ready Apache Airflow DAGs with TaskFlow API and error handling.

Community
Advanced
RDEWAIRDEWAI
5

validate-dag

Validate Airflow DAG files for import errors and coding standards.

Official
Intermediate