skills.rest
RDEWAI
RDEWAIOFFICIAL
Explore all skills by this author →
GitHub

create-pipeline

Generate CI/CD pipeline configurations for medallion data engineering workflows.

5|1|Updated Sep 23, 2025
Search Tags:#automation#devops#gitlab-ci#github-actions#ci-cd#medallion#data-pipeline
One-click install
npx skills add https://github.com/RDEWAI/Redefining-DataEngineering-With-AI --skill create-pipeline-rdewai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: create-pipeline
Source: https://github.com/RDEWAI/Redefining-DataEngineering-With-AI/tree/main/chapter-6/developer-plugin/skills/create-pipeline
Command: npx skills add https://github.com/RDEWAI/Redefining-DataEngineering-With-AI --skill create-pipeline-rdewai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manually writing and maintaining CI/CD pipeline configurations for medallion data engineering pipelines is time-consuming, error-prone, and often misaligned with project-specific coding standards and library version requirements. This Skill eliminates that overhead by automating the generation of production-ready pipeline configs tailored to your project.

Core Features & Use Cases

  • Platform-Agnostic Pipeline Generation: Creates configs for both GitHub Actions and GitLab CI, with support for deployment targets including local, Kubernetes, Astronomer, and MWAA.
  • Project-Aligned Standards: Auto-resolves workspace structure and loads the latest project coding patterns and pinned library versions to ensure generated configs match team requirements.
  • Use Case: A data team building a medallion pipeline with DuckDB and SQLMesh can use this Skill to instantly generate a complete CI/CD workflow that runs linting, unit tests, integration tests, and automated deployment to Astronomer on pushes to the main branch.

Quick Start

Use the create-pipeline skill to generate a GitHub Actions CI/CD workflow for your medallion data pipeline that includes lint, test, and deploy stages.

Frequently Asked Questions about create-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate CI/CD pipelines for medallion data engineering workflows?▼

Automating CI/CD for medallion data pipelines involves generating configuration files that run linting, unit tests, and integration tests before branch-based deployment to targets like Kubernetes or Astronomer.

Can I generate pipeline configurations for both GitHub Actions and GitLab CI?▼

Yes, you can generate CI/CD pipeline configurations for both GitHub Actions and GitLab CI, allowing you to apply automated testing and deployment gating across different DevOps platforms for your data transformation workflows.

How do I ensure my generated CI/CD config matches my project's pinned library versions?▼

To ensure CI/CD configs match pinned library versions, the generation process auto-resolves the workspace structure and loads your latest project coding patterns, applying these standards directly to the reproducible pipeline build.

Does CI/CD automation support deployment to managed Apache Airflow services like MWAA?▼

Yes, CI/CD automation supports deployment to managed Apache Airflow services including MWAA and Astronomer, alongside local and Kubernetes targets, enabling automated data pipeline releases from branch-based gates.

What is the best way to structure CI/CD for a medallion pipeline using DuckDB and SQLMesh?▼

The best way to structure CI/CD for a medallion pipeline is to generate a workflow with distinct lint, test, and deploy stages, ensuring reproducible builds with dependency caching and compliance with project DevOps patterns.

Why should I automate pipeline configuration generation instead of writing YAML manually?▼

Automating pipeline configuration generation eliminates the overhead of manually writing YAML, reducing errors and ensuring alignment with project-specific coding standards, pinned library versions, and reproducible build requirements.

Related Skills

Explore More Data & Analytics Skills→
ActiveInferenceInstituteActiveInferenceInstitute
10

uv-package-manager

Automate Python dependency management and virtual environment setup with uv.

Community
Advanced
astronomerastronomer
490

helm-chart

Develop and validate Helm charts for the Astronomer Private Cloud.

Official
Advanced
cameronraysmithcameronraysmith
14

preferences-schema-versioning

Manage declarative schema versioning and generate type-safe data models for PostgreSQL and DuckDB.

Community
Advanced
LambdaTestLambdaTest
350

cicd-pipeline-skill

Generate CI/CD pipeline configurations for GitHub Actions, Jenkins, GitLab CI, and Azure DevOps.

Official
Advanced