Validate Starlake YAML configuration files against the Starlake JSON Schema.
npx skills add https://github.com/starlake-ai/starlake-skills --skill config-starlake-ai
Please help me install this Agent Skill. Skill: config Source: https://github.com/starlake-ai/starlake-skills/tree/main/skills/config Command: npx skills add https://github.com/starlake-ai/starlake-skills --skill config-starlake-ai
Centralizes expert guidance to create, validate, and harden Starlake YAML configuration so teams avoid misconfigured pipelines, schema drift, and environment leakage.
Ask the skill to validate metadata/application.sl.yml against the Starlake JSON Schema and report missing or invalid fields.
High-intent search queries and answers about installing and using this skill.
Starlake YAML configuration validation involves mapping your application, load, transform, and expectations files against the official Starlake JSON Schema to identify missing mandatory properties like version and application sections, improper variable substitution, and invalid field definitions.
Starlake YAML configuration supports write strategies including SCD2, UPSERT, and OVERWRITE_BY_PARTITION for data pipelines, allowing you to control how incoming data is merged, updated, or replaced across supported destinations like DuckDB, BigQuery, Snowflake, and Redshift.
Starlake YAML configuration uses environment variable conventions and area layouts for incoming, pending, and archive data, ensuring proper storage connector setup for DuckDB, BigQuery, Snowflake, Redshift, and S3 across different environments.
Yes, Starlake YAML configuration is designed to work with both Airflow and Dagster orchestration tooling, applying standardized metadata across application, load, transform, and extract configuration files for your workflows.
Starlake pipelines fail when YAML configuration files lack mandatory properties such as the version field or application sections, or when variable substitution is incorrect, causing schema validation errors during CLI execution or orchestration runs.
To add data quality expectations in Starlake, define rules in your expectations YAML configuration file, applying the correct JSON Schema structure to validate incoming and pending data before it moves through the pipeline transformation and load stages.
rilldataConfigure Rill connectors for diverse data sources with JSON schema.
duggal1Document reusable DAG patterns and retry strategies for ETL/ELT workflows.
joyz0Parse JSON5 config files, substitute environment variables, and validate with Zod schemas.
camilooscargbaptistaAssess data pipelines, ETL/ELT processes, and governance across batch and streaming systems.
