skills.rest
ouakar
ouakarCOMMUNITY
Explore all skills by this author →
GitHub

data-engineer

Automate end-to-end data pipelines from ingestion to analytics-ready datasets.

Updated Mar 25, 2026
Search Tags:#dbt#etl#airflow#data-pipelines#data-quality#data-engineering#elt
One-click install
npx skills add https://github.com/ouakar/ubinarys-dental --skill data-engineer-ouakar
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-engineer
Source: https://github.com/ouakar/ubinarys-dental/tree/main/skills/forgewright/skills/data-engineer
Command: npx skills add https://github.com/ouakar/ubinarys-dental --skill data-engineer-ouakar

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Data engineers need reliable patterns to design, implement, and maintain end-to-end data pipelines and analytics-ready datasets across diverse sources and destinations.

Core Features & Use Cases

  • End-to-end pipeline design and orchestration for ingestion, transformation, and delivery to data warehouses.
  • Data quality framework with validation, lineage, and monitoring across Bronze/Silver/Gold medallion layers.
  • DBT modeling, Spark processing, and orchestration with Airflow or Dagster for scalable analytics engineering.

Quick Start

Set up a minimal end-to-end pipeline by ingesting a sample source, running a simple dbt transformation, and publishing the Gold dataset to your analytics warehouse.

Frequently Asked Questions about data-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build scalable data pipelines with dbt and Airflow?▼

Build scalable data pipelines by orchestrating ingestion and transformation with Airflow, then using dbt for modular SQL modeling to publish analytics-ready datasets to your data warehouse.

What is medallion architecture for ETL pipeline design?▼

Medallion architecture structures data into Bronze, Silver, and Gold layers, applying progressive data quality checks and transformations to produce refined analytics-ready datasets.

How do I ensure idempotent pipelines when processing data with Spark?▼

Ensure idempotent pipelines by designing modular Spark transformations that yield identical results across multiple executions, preventing duplicate records in your data warehouse.

Can I use Dagster instead of Airflow for ELT pipeline orchestration?▼

Yes, Dagster serves as an alternative to Airflow for ELT pipeline orchestration, supporting scalable data infrastructure from initial ingestion to final analytics-ready dataset delivery.

How do I implement data quality checks across medallion architecture layers?▼

Implement data quality checks by applying validation, lineage tracking, and monitoring frameworks across Bronze, Silver, and Gold layers during your dbt transformation workflows.

Does this data engineering approach support Kafka for real-time ingestion?▼

Yes, the end-to-end data infrastructure approach supports Kafka for ingestion, feeding raw data into Spark processing and dbt modeling for scalable analytics engineering.

Related Skills

Explore More Data & Analytics Skills→
rbutinarrbutinar

fabric-sp-run

Execute and monitor stored procedures on a Microsoft Fabric Warehouse.

Community
Intermediate
augustoheissaugustoheiss

agency-data-engineer

Design idempotent ETL/ELT pipelines with medallion lakehouse architectures.

Community
Advanced
Patasse97Patasse97

senior-data-engineer

Design and orchestrate data pipelines with Python, SQL, Spark, Airflow, and dbt.

Community
Advanced
AeyeOpsAeyeOps
3

python-data-engineering

Automate API-to-PostgreSQL data pipelines with SQLAlchemy 2.0 patterns.

Official
Advanced