skills.rest
Phife726
Phife726COMMUNITY
Explore all skills by this author →
GitHub

ds-data-engineering

Build ETL/ELT pipelines, data warehouses, and orchestration for data engineering projects.

Updated Mar 8, 2026
Search Tags:#data#dbt#etl#airflow#pipelines#data-quality#elt
One-click install
npx skills add https://github.com/Phife726/ds_agent --skill ds-data-engineering
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ds-data-engineering
Source: https://github.com/Phife726/ds_agent/tree/main/ds-data-engineering
Command: npx skills add https://github.com/Phife726/ds_agent --skill ds-data-engineering

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Builds the data engineering foundation for DS projects: pipelines, ETL/ELT jobs, data warehouses, orchestration, and data quality checks. It shifts teams from manual, brittle data handling to repeatable, auditable data pipelines that fuel analytics and ML.

Core Features & Use Cases

  • ETL/ELT pipelines, data warehouses, orchestration, and data quality systems
  • Architecture decisions guidance (ETL vs ELT; lake vs warehouse)
  • Incremental loading strategies and governance for reliable data delivery

Quick Start

Define your data sources and targets, choose ETL or ELT, and implement a simple incremental pipeline to load data into your warehouse.

Frequently Asked Questions about ds-data-engineering

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build reliable data pipelines for ETL and ELT workflows?▼

To build reliable data pipelines, define your data sources and targets, choose between ETL or ELT workflows, and implement incremental loading with orchestration, logging, and auditing for repeatable data delivery.

What is the difference between ETL and ELT for data warehousing?▼

The difference between ETL and ELT lies in where data transforms; this Skill guides architectural decisions between them, alongside data lake versus data warehouse choices, to match your project requirements.

How do I implement incremental loading strategies in dbt?▼

Incremental loading strategies in dbt are implemented by defining data sources and targets, then applying governance and orchestration to load only new or updated data into your warehouse reliably.

Can I use Airflow for data quality checks and orchestration?▼

Yes, Airflow supports orchestration for your data pipelines, and this Skill integrates data quality systems with logging and auditing to ensure reliable, auditable data delivery across teams.

Do I need a data lake or a data warehouse for my DS project?▼

Choosing a data lake or data warehouse depends on your architectural needs; this Skill provides architectural decision guidance to help you select the right infrastructure for your DS project.

Related Skills

Explore More Data & Analytics Skills→
clovernguyen1010-ship-itclovernguyen1010-ship-it

ba-data-system-questioning

Generate structured elicitation questions for data-system requirement discovery.

Community
Advanced
omeraltnomeraltn

agency-analytics-reporter

Transform raw business data into analytics reports with SQL and Python.

Community
Advanced
ancolemanancoleman
503

transforming-data

Transform raw data into analytical assets using ETL/ELT patterns and orchestration tools.

Community
Advanced
chicanoandres702chicanoandres702

data-engineering-data-pipeline

Design and orchestrate scalable batch and streaming data pipelines.

Community
Advanced