data-pipeline

Automates ETL/ELT workflows with counting, transformation, verification, and monitoring phases.

2|1|Updated Mar 8, 2026

Search Tags:#workflow #monitoring #validation #orchestration #etl #data-quality #data-pipeline

Ready to run

npx skills add https://github.com/duggal1/Sapphire-cli --skill data-pipeline-duggal1

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-pipeline
Source: https://github.com/duggal1/Sapphire-cli/tree/main/skills/data-pipeline/.claude/skills/data-pipeline
Command: npx skills add https://github.com/duggal1/Sapphire-cli --skill data-pipeline-duggal1

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Data engineers need a disciplined framework to design, orchestrate, and monitor complex data pipelines that span counting, transformation, verification, and ongoing monitoring.

Core Features & Use Cases

Phase-based workflow setup, workspace scaffolding, and role definitions (etl-architect, data-quality-manager, scheduler-engineer, monitoring-specialist, pipeline-reviewer).
End-to-end pipeline orchestration across ETL/ELT, with templates and guidance for building robust data flows.
Use cases include planning new pipelines, validating data quality, scheduling jobs, and generating final review reports.

Quick Start

Create the _workspace/ project root, place input data in _workspace/00_input.md, and begin the three-phase pipeline workflow described above.

Frequently Asked Questions about data-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I orchestrate end-to-end ETL workflows with data quality validation?▼

End-to-end ETL orchestration is automated through a structured, phase-based workflow that handles counting, transformation, verification, and monitoring. It assigns defined roles like data-quality-manager and monitoring-specialist to coordinate cross-role execution across workspace phases.

What is the best way to monitor data pipeline SLAs and ensure data quality?▼

Monitoring data pipeline SLAs is handled by a dedicated monitoring-specialist role within the workflow. Data quality is ensured through a data-quality-manager role that performs verification and validation across the pipeline phases.

How do I set up a new data pipeline project for automated transformation and verification?▼

To set up a data pipeline project, create the `_workspace/` project root, place input data in `_workspace/00_input.md`, and begin the three-phase pipeline workflow. The Skill provides workspace scaffolding and reusable templates for the pipeline structure.

Can I use this pipeline orchestration framework for both ETL and ELT workflows?▼

Yes, the pipeline orchestration framework applies to both ETL and ELT workflows. It automates the full pipeline lifecycle including transformation, verification, and monitoring regardless of the specific data integration pattern.

What roles are defined for coordinating cross-role execution in data pipeline workflows?▼

The data pipeline workflow defines five specialized roles: etl-architect, data-quality-manager, scheduler-engineer, monitoring-specialist, and pipeline-reviewer. These roles coordinate cross-role execution across the structured workspace phases.

Does this data pipeline orchestration tool require any external dependencies?▼

No, this data pipeline orchestration tool has no external dependencies. It operates independently using its own internal workspace scaffolding and templates to manage the ETL workflow phases.