skills.rest
siddharthchauhan
siddharthchauhanCOMMUNITY
Explore all skills by this author →
GitHub

pipeline-orchestration

Orchestrate the 7-phase SDTM ETL pipeline with subagent delegation and human checkpoints.

Updated Aug 27, 2026
Search Tags:#validation#subagents#ETL#pipeline#orchestrate#SDTM#Define.xml
One-click install
npx skills add https://github.com/siddharthchauhan/ETL --skill pipeline-orchestration-siddharthchauhan
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: pipeline-orchestration
Source: https://github.com/siddharthchauhan/ETL/tree/main/sdtm_pipeline/deepagents/skills/pipeline-orchestration
Command: npx skills add https://github.com/siddharthchauhan/ETL --skill pipeline-orchestration-siddharthchauhan

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

The Pipeline Orchestration skill provides a centralized controller for the complete 7-phase SDTM ETL pipeline, managing subagent delegation, checkpoints, self-correction loops, and pipeline state to ensure consistent, submission-ready SDTM data.

Core Features & Use Cases

  • Orchestrates the end-to-end 7-phase SDTM ETL workflow from data ingestion to data warehouse loading.
  • Delegates domain-specific transformations, validation, code generation, and data loading to specialized subagents, improving reliability and scalability.
  • Supports human review checkpoints, self-correction loops (up to 3 iterations), and detailed pipeline state tracking for auditability.
  • Use Case: A clinical data team runs MAXIS-08 through the pipeline to produce Define.xml and SDTM outputs with validation at multiple stages.

Quick Start

Orchestrate the full pipeline for a study by planning tasks, loading source data from S3, and executing phases 1–7 with appropriate delegations.

Frequently Asked Questions about pipeline-orchestration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I orchestrate an end-to-end SDTM ETL pipeline for clinical data?▼

You orchestrate an SDTM ETL pipeline by executing a 7-phase workflow that covers data ingestion, validation, transformation, and Define.xml generation. The pipeline delegates domain-specific tasks to subagents while managing state and checkpoints.

What is the best way to automate Define.xml generation during SDTM transformation?▼

Automating Define.xml generation is handled as a dedicated phase within the 7-phase SDTM ETL pipeline. The orchestrator delegates code generation and validation to specialized subagents, ensuring submission-ready outputs.

Does SDTM pipeline orchestration support self-correction loops for validation failures?▼

SDTM pipeline orchestration supports self-correction loops with up to 3 iterations. When validation fails, the pipeline automatically triggers corrections before passing data through human review checkpoints.

Can I use subagents to delegate domain-specific transformations in an ETL pipeline?▼

You can delegate domain-specific transformations to specialized subagents within the ETL pipeline. This delegation pattern improves reliability and scalability across the ingestion, transformation, and data warehouse loading phases.

How does pipeline state management work for multi-phase clinical data processing?▼

Pipeline state management tracks execution progress across all 7 SDTM ETL phases for auditability. It maintains state through subagent delegations, self-correction loops, and human review checkpoints to ensure consistent outputs.

What are the limitations of using orchestration for SDTM data transformation?▼

A limitation of SDTM pipeline orchestration is that self-correction loops are capped at 3 iterations. If validation issues persist beyond this threshold, the pipeline relies on human review checkpoints to resolve data discrepancies.

Related Skills

Explore More Data & Analytics Skills→
kubiyabotkubiyabot
16

aws-skill

Manage AWS S3, EC2, and Lambda resources from the terminal.

Official
Advanced
mark64oswaldmark64oswald
1

healthsim-trialsim

Generate regulatory-grade synthetic clinical trial data across SDTM domains.

Community
Advanced
LiuHarry1LiuHarry1
4

create-subagent

Create specialized subagents with YAML frontmatter name and description for isolated AI tasks.

Community
Advanced
sancovpsancovp

deep-research-accumulate

Research external systems via web searches and repository clones into cited markdown files.

Community
Advanced