data-engineering

Name: data-engineering
Availability: InStock
Author: EloPhanto

Build end-to-end data pipelines transforming raw data into analytics-ready assets.

116|9|Updated Feb 18, 2026

Search Tags:#dbt #etl #spark #data-pipelines #data-quality #data-engineering

Ready to run

npx skills add https://github.com/elophanto/EloPhanto --skill data-engineering-elophanto

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-engineering
Source: https://github.com/elophanto/EloPhanto/tree/main/skills/data-engineering
Command: npx skills add https://github.com/elophanto/EloPhanto --skill data-engineering-elophanto

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Expert data engineers design, build, and operate scalable data infrastructures that transform raw data into analytics-ready assets, ensuring reliability, observability, and governance.

Core Features & Use Cases

End-to-end data pipelines (Bronze -> Silver -> Gold) with idempotent processing and strong data contracts.
Platform-agnostic architecture: Spark, Delta Lake, dbt, and streaming for batch and real-time workloads.
Data quality, lineage, and observability with schema validation, monitoring, and alerting for SLAs.

Quick Start

Set up an end-to-end data pipeline from Bronze to Gold using Spark, Delta Lake, and dbt with automated quality checks.

Frequently Asked Questions about data-engineering

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I build end-to-end data pipelines with Bronze-Silver-Gold architecture?▼

End-to-end data pipelines transform raw data through Bronze-Silver-Gold layers into analytics-ready assets. This architecture applies idempotent processing and explicit schema contracts to ensure reliable, structured data delivery across each tier.

How does data lineage and observability work in a lakehouse architecture?▼

Data lineage and observability in a lakehouse architecture track data origins and transformations while monitoring schema validation and SLA alerting. This mechanism ensures strong data quality controls and reliable analytics across streaming and batch workloads.

Can I use dbt and Spark together for batch and real-time streaming workloads?▼

Yes, dbt and Spark work together for batch and real-time streaming workloads in a platform-agnostic architecture. This combination supports Delta Lake storage and automated quality checks across major cloud platforms for both processing modes.

What is the best way to enforce data contracts and schema validation in ETL pipelines?▼

The best way to enforce data contracts and schema validation in ETL pipelines is through idempotent processing combined with automated quality checks. This method satisfies explicit schema requirements and prevents invalid data from progressing downstream.

When do I need idempotent processing for data quality and pipeline resilience?▼

You need idempotent processing for data quality and pipeline resilience when running end-to-end data pipelines that require safe retries. It ensures replaying transformations does not duplicate or corrupt analytics-ready assets stored in your lakehouse.