eval-pipeline

What problem does it solve?

The eval pipeline provides a structured, end-to-end workflow to generate, grade, review, and report on AI-agent prompts, enabling consistent evaluation and faster debugging of prompts and agents.

Core Features & Use Cases

• End-to-end orchestration of generation, multi-model grading, reviewer feedback, and report generation for AI prompts.
• Provides detailed action timelines and workspace isolation to ensure reproducibility and auditability.
• Use Case: Run a full evaluation for a given prompt to identify strengths and weaknesses across generation quality, adherence to guidelines, and safety considerations.

Quick Start

Run an evaluation pipeline that generates code, grades it using multiple graders, and compiles a final report for a selected prompt.