data-pipeline-engineering

Automate ETL pipelines for pilot analytics projects.

1|Updated Jan 20, 2026

Search Tags:#analytics #orchestration #pipeline #transformation #etl #data-quality #data-ingestion

Ready to run

npx skills add https://github.com/profmikegreene/gotei --skill data-pipeline-engineering

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-pipeline-engineering
Source: https://github.com/profmikegreene/gotei/tree/main/Gotei_Skills/data-pipeline-engineering
Command: npx skills add https://github.com/profmikegreene/gotei --skill data-pipeline-engineering

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Develop and manage data ingestion, processing, and transformation pipelines for pilot projects. Use when automating ETL workflows, integrating new data sources, or building canonical datasets to support downstream analytics.

Core Features & Use Cases

Automated ETL Orchestration: Automate end-to-end data pipelines from source ingestion to canonical dataset creation.
Data Source Integration: Connect and harmonize multiple data sources for unified analytics.
Use Case: Build a repeatable data pipeline that ingests customer data, applies schema mappings, and outputs a canonical dataset for BI dashboards.

Quick Start

Instruct the AI to scaffold an end-to-end ETL pipeline for pilot data sources and target a canonical warehouse.

Frequently Asked Questions about data-pipeline-engineering

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate an ETL pipeline for ingesting and transforming pilot data?▼

Automate an ETL pipeline by scaffolding end-to-end orchestration that ingests data sources, applies schema mapping, validates data quality, and outputs a canonical dataset. The pipeline includes lightweight error handling and retry logic for reliable processing.

What is the best way to integrate multiple data sources for a unified analytics dataset?▼

Integrating multiple data sources for unified analytics requires harmonizing schemas and applying validation rules to generate a canonical dataset. This Skill automates that source integration and transformation process specifically for pilot project environments.

Can I use this ETL automation approach for building canonical datasets in pilot environments?▼

Yes, you can use this ETL automation approach for pilot environments. It is designed to satisfy requirements for lightweight orchestration, data source integration, and canonical dataset generation without heavy infrastructure commitments.

How do I handle schema mapping and data validation errors during data ingestion?▼

Handle schema mapping and data validation errors during data ingestion using built-in error handling and retry logic. The pipeline validates data quality automatically as it processes and transforms records for the downstream canonical warehouse.

Does this data pipeline orchestration require complex dependencies or external components?▼

No, this data pipeline orchestration does not require complex dependencies or external components. It operates as a standalone Skill with no listed dependencies, making it easy to start scaffolding ETL workflows immediately.

When should I avoid using lightweight ETL orchestration for my data pipeline?▼

You should avoid using lightweight ETL orchestration if your project requires heavy enterprise-grade infrastructure or complex real-time streaming beyond pilot analytics. This approach is specifically scoped for lightweight pilot environments and canonical dataset generation.