skills.rest
Lordjiggy
LordjiggyCOMMUNITY
Explore all skills by this author →
GitHub

data-engineer

Automate ETL pipeline design, orchestration, and data warehouse schema modeling.

Updated Aug 27, 2026
Search Tags:#analytics#data quality#ETL#data engineering#pipeline#data warehouse#SQL optimization
One-click install
npx skills add https://github.com/Lordjiggy/claude-code-marketplace --skill data-engineer
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-engineer
Source: https://github.com/Lordjiggy/claude-code-marketplace/tree/main/skills/agents/data-engineer
Command: npx skills add https://github.com/Lordjiggy/claude-code-marketplace --skill data-engineer

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Designing and implementing efficient ETL pipelines, data warehouses, and analytics infrastructure requires specialized data engineering expertise, which can be complex and time-consuming. This Skill provides an expert agent to streamline your data workflows.

Core Features & Use Cases

  • ETL Pipeline Design: Builds Extract, Transform, Load (ETL) workflows using tools like Apache Airflow or Dagster.
  • Data Warehousing: Designs star/snowflake schemas for data warehouses (Snowflake, BigQuery, Redshift).
  • Data Quality & Analytics: Sets up validation rules, anomaly detection, and optimizes SQL queries for reporting.
  • Use Case: Design an ETL pipeline for customer data, create a data warehouse schema for business intelligence, build a real-time analytics dashboard, or optimize slow SQL queries for reporting.

Quick Start

Design ETL pipeline for customer data.

Frequently Asked Questions about data-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an ETL pipeline for data warehousing?▼

ETL pipeline design automates extraction, transformation, and loading of data into warehouses using orchestration tools like Airflow or Dagster. Define source connectors, apply transformation logic, validate data quality, and load into schemas (star or snowflake) on platforms like Snowflake, BigQuery, or Redshift for analytics delivery.

What's the best way to optimize SQL queries for analytics reporting?▼

SQL optimization for analytics involves tuning queries against data warehouse schemas through indexing, query rewriting, and execution plan analysis. This reduces latency for reporting dashboards and business intelligence queries on large datasets in Snowflake, BigQuery, or Redshift.

Can I build real-time data pipelines with streaming tools like Kafka and Flink?▼

Real-time processing pipelines ingest data via Kafka or Flink, apply transformations, validate quality, and load into analytics platforms. This enables low-latency dashboards and event-driven analytics workflows beyond batch ETL.

How do I set up data quality validation in my ETL workflows?▼

Data quality validation embeds checks during ETL transformation to detect anomalies, missing values, and schema violations before loading into the warehouse. Orchestration tools like Airflow flag failures and ensure only clean data reaches analytics systems.

Does star schema or snowflake schema work better for my data warehouse?▼

Star schema denormalizes dimensions into a flat structure for query simplicity; snowflake schema normalizes further to reduce redundancy. Choice depends on query patterns, storage constraints, and analytics complexity—both are supported in modern data platforms.

Related Skills

Explore More Data & Analytics Skills→
ilyasibrahimilyasibrahim
80

etl-patterns

Standardize Somali dialect data patterns across ETL pipeline stages.

Community
Advanced
weirdgmeweirdgme
4

platform-engineering

Design and implement scalable platform engineering patterns for internal developer platforms.

Community
Advanced
wzyxdwllwzyxdwll
14

data-engineering

Automate data pipeline design, orchestration, and quality checks across Airflow, Dagster, and Prefect.

Community
Advanced
Patasse97Patasse97

senior-data-engineer

Design and orchestrate data pipelines with Python, SQL, Spark, Airflow, and dbt.

Community
Advanced