data-engineering-master

What problem does it solve?

Data Engineering work often stalls on unclear design choices, fragile pipelines, missing quality gates, and runaway cost. This Skill helps you think like a senior data engineering practitioner to design and deliver reliable, queryable, trustworthy data platforms end-to-end.

Core Features & Use Cases

• Pipeline and platform OS: Covers the full lifecycle from ingestion to serving, including security/data management, DataOps, data architecture, orchestration, and software-engineering practices.
• Decision-ready engineering playbooks: Uses an agentic protocol to structure research around data scale/engine fit, build-vs-buy ingestion, modeling and consumption patterns, batch-vs-streaming boundaries, table/lakehouse formats, data quality & contracts, and cost & governance.
• Workflow SOP walkthroughs: Provides guided checklists for ELT construction, CDC/incremental ingestion, dbt dimensional modeling structure, DAG orchestration, lakehouse table landing, data quality/contract gates, and DataOps CI/CD + FinOps.

Quick Start

Trigger the data-engineering-master skill by asking your question with relevant terms like "data engineering" and include what you are building (e.g., "I need to design a CDC pipeline from MySQL using Debezium into a lakehouse table with dbt transforms and strict data quality gates").