Execute named data transformation tasks in Starlake pipelines with YAML definitions.
npx skills add https://github.com/starlake-ai/starlake --skill transform-starlake-ai
Please help me install this Agent Skill. Skill: transform Source: https://github.com/starlake-ai/starlake/tree/main/.agent/skills/transform Command: npx skills add https://github.com/starlake-ai/starlake --skill transform-starlake-ai
Transform tasks in data pipelines often require manual scripting or ad-hoc SQL work. This skill provides a standard, declarative way to run predefined transformation steps within Starlake without writing custom code each time.
Run a transformation task by specifying its name, for example: starlake transform --name mydomain.mytask
High-intent search queries and answers about installing and using this skill.
You can run data transformation tasks declaratively by executing a single command with the task name. This requires YAML-based task definitions integrated with the Starlake transform workflow to parse options and execute the specified task.
Yes, you can validate SQL generation before execution using the dry-run option. This allows you to preview the transformation output and ensure SQL accuracy within your Starlake pipeline without applying actual changes.
A YAML-based task runner handles recursive dependencies by parsing task definitions to identify and execute prerequisite transformations in the correct order. This ensures all dependent tasks are resolved before the target task runs.
You can override queries during ETL data transformations by using the --query option. This allows you to specify custom SQL directly, replacing the default query defined in your YAML-based task configuration.
Yes, you must define task names to automate routine data transformations. Using the --name option with a structured identifier like mydomain.mytask ensures consistent execution across different environments.
The best way to execute scheduled data transformations is by using the --scheduledDate option. This parameter allows you to specify a particular execution date, ensuring the transformation task processes the correct data partition.
starlake-aiIndex Starlake CLI skills for command references and pipeline workflows.
christopherlouetDesign and manage automated ETL/ELT data pipelines using Python, SQL, Airflow, and dbt.
zjykzkIngest raw text data into an Obsidian wiki as searchable knowledge.
4lermanBuild scalable data pipelines with Python, SQL, Spark, and Airflow.
