Pipeline Orchestration

Coordinates prompt assembly, output validation, retries, and logging across pipeline stages.

Updated Apr 11, 2026
One-click install
npx skills add https://github.com/gozonerd/shadow-ai-assessment --skill pipeline-orchestration-gozonerd
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: Pipeline Orchestration
Source: https://github.com/gozonerd/shadow-ai-assessment/tree/main/.claude/skills/dmis/task_type/TT-28_Pipeline_Orchestration
Command: npx skills add https://github.com/gozonerd/shadow-ai-assessment --skill pipeline-orchestration-gozonerd

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, and includes scripts (resource) and references (resource) components.

What problem does it solve? Running multi-stage AI pipelines with dozens of parallel threads makes it hard to track which prompts were assembled, which outputs passed schema validation, when retries occurred, and whether stakeholders were notified. This Skill standardizes that orchestration layer so every stage produces a complete, auditable execution log. ## Core Features & Use Cases - Prompt Assembly & Dispatch: Builds stage-specific prompts from skill definitions, input context, and prior outputs, then dispatches them to task type agents with configured thread counts (e.g., 25 threads for P2, 16 threads across 6 passes for P6). - Schema Validation & Retry Control: Validates every output against the correct task type schema and enforces a strict maximum of one retry per failure, logging remediation details for each validation error. - Monitoring, Notification & Logging: Records timestamped execution events, delivers status notifications to configured recipients, and produces a completion summary with timing metrics in YAML conforming to the TT-28 output schema. - Use Case: While running pipeline P2, 24 of 25 synthesis outputs pass validation; the orchestrator retries the failed thread once, confirms the retry passes, notifies the QA team, and writes a full orchestration log. ## Quick Start Ask the AI to orchestrate pipeline stage P2 by assembling prompts, dispatching threads, validating outputs against the task type schema, and producing the TT-28 orchestration log.

Frequently Asked Questions about Pipeline Orchestration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I orchestrate multi-stage AI pipelines with parallel threads?

Assemble stage-specific prompts from skill definitions and prior outputs, dispatch them to task type agents with the configured thread count, then validate all outputs against the task type schema. Record every event in a timestamped execution log and produce a completion summary with timing metrics.

How do I validate LLM pipeline outputs against a YAML schema?

Apply the task type's output schema to each thread output and log pass/fail counts per task type. The included validate_output.py script checks required fields, valid action and status enums, retry limits, and notification delivery in the orchestration log.

What retry policy should pipeline orchestration enforce?

This orchestration spec enforces a maximum of one retry per failed output, per the audit layer specification. Each retry is logged with the original failure, retry count, and final status of resolved or unresolved; retrying beyond the limit is flagged as an error by the validator.

Why does pipeline output validation fail on some threads?

Failures typically come from missing required log fields, invalid action or status values, or outputs that violate the task type schema such as missing divergence log entries or incorrect reference mappings. Each failure should be logged with remediation details and retried once with revised context.

What are the limitations of schema-based pipeline orchestration?

Schema validation checks structure and completeness but cannot verify semantic correctness of generated content. It also depends on correct schema selection per task type; validating against the wrong or outdated schema produces misleading pass results, so accuracy rules and audit layers remain necessary.