skills.rest
mikeparcewski
mikeparcewskiCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

pipeline

Design, analyze, and optimize batch, streaming, and incremental ETL pipelines.

8|2|Updated Feb 9, 2026
Search Tags:#architecture#performance#orchestration#observability#pipeline#data-quality#data-etl
One-click install
npx skills add https://github.com/mikeparcewski/wicked-garden --skill pipeline-mikeparcewski
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pipeline
Source: https://github.com/mikeparcewski/wicked-garden/tree/main/skills/data/pipeline
Command: npx skills add https://github.com/mikeparcewski/wicked-garden --skill pipeline-mikeparcewski

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps teams design, review, and optimize data pipelines and ETL workflows to improve reliability, scalability, and performance.

Core Features & Use Cases

  • Design pipelines across batch, streaming, and incremental ETL with architecture diagrams and orchestration plans.
  • Review existing pipelines for quality, maintainability, and performance.
  • Use cases: Transitioning to modern orchestration tools, establishing observability and governance for data workflows.

Quick Start

Design a daily batch pipeline from a source database to a data lake with monitoring.

Frequently Asked Questions about pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a data pipeline with batch and streaming ETL?▼

Designing a data pipeline with batch and streaming ETL requires applying architecture templates and orchestration plans to structure data flow. This approach establishes clear deployment templates across diverse data sources and tools.

How do I add observability and quality gates to my ETL workflows?▼

Adding observability and quality gates to ETL workflows involves enforcing monitoring patterns within your data pipelines. This ensures data reliability and maintainability across incremental processing loads.

What is the best way to review existing data pipelines for performance?▼

The best way to review existing data pipelines for performance is to analyze them for reliability, scalability, and maintainability. This process identifies optimization opportunities within your current ETL workflows.

Does this approach support transitioning to modern data orchestration tools?▼

Yes, this approach supports transitioning to modern data orchestration tools by providing architecture diagrams and governance patterns. It enables teams to establish robust workflows across diverse data sources.

Can I use this to plan a daily batch pipeline from a database to a data lake?▼

Yes, you can plan a daily batch pipeline from a source database to a data lake. This skill provides architecture templates and monitoring plans to govern the complete ETL workflow.

Related Skills

Explore More Data & Analytics Skills→
Snowflake-LabsSnowflake-Labs
1

convert

Convert SQL from multiple database platforms into Snowflake syntax.

Official
Intermediate
librefanglibrefang
10

data-pipeline

Automates production-grade data pipeline design and operation for Spark, Airflow, and dbt workflows.

Official
Intermediate
ClementATHClementATH

omc-reference

Load a centralized catalog of OMC agents, tools, and workflows.

Community
Intermediate
chicanoandres702chicanoandres702

data-engineering-data-pipeline

Design and orchestrate scalable batch and streaming data pipelines.

Community
Advanced