pipeline

Design, analyze, and optimize batch, streaming, and incremental ETL pipelines.

8|2|Updated Feb 9, 2026

Search Tags:#architecture #performance #orchestration #observability #pipeline #data-quality #data-etl

Ready to run

npx skills add https://github.com/mikeparcewski/wicked-garden --skill pipeline-mikeparcewski

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: pipeline
Source: https://github.com/mikeparcewski/wicked-garden/tree/main/skills/data/pipeline
Command: npx skills add https://github.com/mikeparcewski/wicked-garden --skill pipeline-mikeparcewski

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps teams design, review, and optimize data pipelines and ETL workflows to improve reliability, scalability, and performance.

Core Features & Use Cases

Design pipelines across batch, streaming, and incremental ETL with architecture diagrams and orchestration plans.
Review existing pipelines for quality, maintainability, and performance.
Use cases: Transitioning to modern orchestration tools, establishing observability and governance for data workflows.

Quick Start

Design a daily batch pipeline from a source database to a data lake with monitoring.

Frequently Asked Questions about pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I design a data pipeline with batch and streaming ETL?▼

Designing a data pipeline with batch and streaming ETL requires applying architecture templates and orchestration plans to structure data flow. This approach establishes clear deployment templates across diverse data sources and tools.

How do I add observability and quality gates to my ETL workflows?▼

Adding observability and quality gates to ETL workflows involves enforcing monitoring patterns within your data pipelines. This ensures data reliability and maintainability across incremental processing loads.

What is the best way to review existing data pipelines for performance?▼

The best way to review existing data pipelines for performance is to analyze them for reliability, scalability, and maintainability. This process identifies optimization opportunities within your current ETL workflows.

Does this approach support transitioning to modern data orchestration tools?▼

Yes, this approach supports transitioning to modern data orchestration tools by providing architecture diagrams and governance patterns. It enables teams to establish robust workflows across diverse data sources.

Can I use this to plan a daily batch pipeline from a database to a data lake?▼

Yes, you can plan a daily batch pipeline from a source database to a data lake. This skill provides architecture templates and monitoring plans to govern the complete ETL workflow.