dag-generate

Generate Airflow DAGs from Starlake project YAML definitions.

209|30|Updated Sep 18, 2021

Search Tags:#automation #workflow #orchestration #airflow #generate #dag #starlake

Ready to run

npx skills add https://github.com/starlake-ai/starlake --skill dag-generate

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: dag-generate
Source: https://github.com/starlake-ai/starlake/tree/main/.agent/skills/dag-generate
Command: npx skills add https://github.com/starlake-ai/starlake --skill dag-generate

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill automates the generation of Airflow DAGs from a Starlake project, enabling consistent orchestration without manual DAG coding.

Core Features & Use Cases

Generates Airflow DAGs from the project's YAML definitions and domain/task configurations.
Supports filtering by tags, tasks, and domains, and can clean or reuse an existing output directory.
Optionally generates role definitions to align DAGs with access controls.

Quick Start

Run starlake dag-generate with an output directory to generate all DAGs for your project.

Frequently Asked Questions about dag-generate

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I generate Airflow DAGs from a Starlake project?▼

To generate Airflow DAGs from a Starlake project, run the starlake dag-generate command with a specified output directory. This automates DAG creation by reading your project's YAML definitions and domain configurations for consistent orchestration without manual coding.

Can I filter which Airflow DAGs are generated by tags, tasks, or domains?▼

Yes, you can selectively generate Airflow DAGs by filtering based on specific tags, tasks, and domains. This controlled inclusion ensures you only deploy the relevant data pipeline workflows to your specified output directory.

What is the best way to automate data pipeline orchestration without writing Airflow DAGs manually?▼

Automating Airflow DAG generation from Starlake YAML definitions eliminates manual DAG coding for data pipeline orchestration. It automatically translates your existing domain and task configurations into deployable Airflow DAG files.

Does dag-generate support cleaning an existing output directory before deployment?▼

Yes, dag-generate supports optional cleaning of an existing output directory before generating new Airflow DAGs. You can choose to clean or reuse the current directory to manage your deployment output effectively.

Can I generate role definitions alongside Airflow DAGs for access control?▼

Yes, you can optionally generate role definitions alongside your Airflow DAGs. This feature aligns your generated DAGs with access controls to maintain proper security roles within your orchestration deployment.