starlake-aiOfficial·42 Agent Skills Included

starlake

Declarative YAML data pipelines for any warehouse

Builds extract, load, transform, and orchestrate data pipelines from simple YAML configuration files. Replaces hundreds of lines of custom ETL code with declarative schemas, merge strategies, and data quality rules. Supports BigQuery, Snowflake, Redshift, DuckDB, PostgreSQL, Delta Lake, and Iceberg with automatic schema inference and lineage tracking. Generates ready-to-run DAGs for Airflow, Dagster, and Snowflake Tasks without writing orchestration code.

209|30

↗

npx skills add starlake-ai/starlake --all -g -y

Available:

Explains the Starlake codebase architecture, command pattern, and YAML metadata layout so the agent can navigate the project and run or extend pipeline commands correctly.

All Skills in This Repository (42)

Pure Emerald Level Indicators

📦 In Repo

starlake-ai

kafkaload

Transfer data between Kafka topics and file storage in Parquet, JSON, and CSV formats.

dag-generate

Generate Airflow DAGs from Starlake project YAML definitions.

yml2ddl

Convert Starlake YML definitions into SQL DDL statements.

metrics

Compute table metrics such as row counts and null counts.

settings

Test named connection settings and health across environments.

extract-bq-schema

Extract BigQuery schemas from Google Cloud projects using CLI options.

extract-script

Generate data extraction scripts from Mustache templates.

validate

Validate Starlake project configurations and data source connections.

infer-schema

Derive structured schemas from CSV, JSON, and XML data files.

dag-deploy

Deploy generated DAGs to a target directory for Airflow or Dagster.

col-lineage

Generates column-level lineage across tasks as JSON for metadata systems.

table-dependencies

Generate a dependency graph of data warehouse tables with SVG/PNG outputs.

Official

Intermediate

Frequently Asked Questions

FAQPage Schema

How to install Starlake?▼

Run `npx skills add starlake-ai/starlake --all -g -y` in your terminal to install all skills in this suite globally.

What does Starlake do?▼

Starlake lets you define entire data pipelines in YAML instead of writing custom ETL code. It handles extraction, loading, SQL transformations, data quality checks, and DAG generation automatically.

Which data warehouses does Starlake support?▼

It works with BigQuery, Snowflake, Redshift, DuckDB, PostgreSQL, and Spark, plus lake formats like Delta Lake, Iceberg, and Parquet.

Can Starlake generate Airflow DAGs automatically?▼

Yes. Starlake reads SQL dependencies from your transformations and generates ready-to-use DAGs for Airflow, Dagster, or Snowflake Tasks with no custom orchestration code.

Do I need to write code to build pipelines with Starlake?▼

No. You declare sources, schemas, merge strategies, and quality rules in YAML files, and Starlake figures out how to execute the pipeline on your chosen warehouse.