dag-deploy

Deploy generated DAGs to a target directory for Airflow or Dagster.

209|30|Updated Sep 18, 2021
One-click install
npx skills add https://github.com/starlake-ai/starlake --skill dag-deploy
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: dag-deploy
Source: https://github.com/starlake-ai/starlake/tree/main/.agent/skills/dag-deploy
Command: npx skills add https://github.com/starlake-ai/starlake --skill dag-deploy

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Deploys generated DAGs to a target directory to enable orchestration by your scheduler.

Core Features & Use Cases

  • Deploy DAGs to a target directory for orchestration tools like Airflow or Dagster.
  • Configure per-project DAG sub-directories and optional cleanup before deployment.
  • Use cases include promoting debug DAGs to a test environment and deploying production DAGs to a central repository.

Quick Start

Run starlake dag-deploy with a specified outputDir to deploy the generated DAGs.

Frequently Asked Questions about dag-deploy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy generated DAGs to an Airflow or Dagster target directory?

To deploy generated DAGs, you run a deployment command that moves DAG files from a specified input directory to an output path for orchestration systems like Airflow or Dagster. This prepares the DAGs for scheduling.

Can I configure per-project DAG sub-directories and clean the output path before deployment?

Yes, you can configure per-project DAG sub-directories within the output path and use a cleanup option to clear the target directory before deployment, ensuring a safe and repeatable DAG deployment process.

What is the best way to promote debug DAGs to a test environment for orchestration?

The best way to promote debug DAGs to a test environment is to deploy them from an input directory to a specified output directory. This action aligns the DAGs with your orchestration system's expected path for testing.

Does the DAG deployment process validate input and output directory options?

Yes, the DAG deployment process validates the usage of options including inputDir, outputDir, dagDir, and clean. This validation aligns with instructions and ensures a safe, repeatable deployment to your orchestration target.

Why do I need to specify an output directory when deploying DAGs for orchestration?

You need to specify an output directory because orchestration tools like Airflow or Dagster require DAGs to be placed in a specific target path. Deploying to this directory enables the scheduler to detect and run the DAGs.