dag-deploy

Deploy generated DAGs to a target directory for Airflow or Dagster.

209|30|Updated Sep 18, 2021

Search Tags:#automation #deployment #orchestration #airflow #dag #dag-deploy

Ready to run

npx skills add https://github.com/starlake-ai/starlake --skill dag-deploy

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: dag-deploy
Source: https://github.com/starlake-ai/starlake/tree/main/.agent/skills/dag-deploy
Command: npx skills add https://github.com/starlake-ai/starlake --skill dag-deploy

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Deploys generated DAGs to a target directory to enable orchestration by your scheduler.

Core Features & Use Cases

Deploy DAGs to a target directory for orchestration tools like Airflow or Dagster.
Configure per-project DAG sub-directories and optional cleanup before deployment.
Use cases include promoting debug DAGs to a test environment and deploying production DAGs to a central repository.

Quick Start

Run starlake dag-deploy with a specified outputDir to deploy the generated DAGs.

Frequently Asked Questions about dag-deploy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I deploy generated DAGs to an Airflow or Dagster target directory?▼

To deploy generated DAGs, you run a deployment command that moves DAG files from a specified input directory to an output path for orchestration systems like Airflow or Dagster. This prepares the DAGs for scheduling.

Can I configure per-project DAG sub-directories and clean the output path before deployment?▼

Yes, you can configure per-project DAG sub-directories within the output path and use a cleanup option to clear the target directory before deployment, ensuring a safe and repeatable DAG deployment process.

What is the best way to promote debug DAGs to a test environment for orchestration?▼

The best way to promote debug DAGs to a test environment is to deploy them from an input directory to a specified output directory. This action aligns the DAGs with your orchestration system's expected path for testing.

Does the DAG deployment process validate input and output directory options?▼

Yes, the DAG deployment process validates the usage of options including inputDir, outputDir, dagDir, and clean. This validation aligns with instructions and ensures a safe, repeatable deployment to your orchestration target.

Why do I need to specify an output directory when deploying DAGs for orchestration?▼

You need to specify an output directory because orchestration tools like Airflow or Dagster require DAGs to be placed in a specific target path. Deploying to this directory enables the scheduler to detect and run the DAGs.