pipeline-architecture-patterns
CommunityGuide data pipelines with proven patterns.
Authorbaekenough
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Data engineering teams need consistent, scalable blueprinting for ETL/ELT pipelines, orchestration, and data quality practices to reduce rework and errors.
Core Features & Use Cases
- Provides Lambda, Kappa, and Medallion architecture guidance for batch and streaming processing.
- Defines data quality contracts, idempotent upserts, and lineage strategies to improve reliability and observability.
- Use Case: design a multi-region data lake with bronze-silver-gold layers and event-driven orchestration.
Quick Start
Review your current pipelines against the Lambda, Kappa, and Medallion patterns and map them to bronze-silver-gold layers to begin modernization.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: pipeline-architecture-patterns Download link: https://github.com/baekenough/second-brain/archive/main.zip#pipeline-architecture-patterns Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 510,000+ vetted skills library on demand.