What problem does it solve?
Iterating an AI agent until its output is good enough often becomes an unbounded, subjective loop with no clear stopping criteria. This Skill structures that process for Claude Managed Agents by requiring a markdown rubric, clamping iteration budgets, reading grader verdicts, and validating generalization with held-back eval cases.
Core Features & Use Cases
- Outcome Payload Builder: Generates a user.define_outcome JSON payload with a required rubric and max_iterations clamped to 1..20, sourced from a build sheet or command-line arguments.
- Verdict Reader: Parses grader results and recommends the next move — SHIP, SHARPEN, ESCALATE, or RESUME — with budget-aware advice when one iteration remains.
- Held-Back Eval Scaffold: Creates parallel eval run plans (capped at the 25-thread CMA ceiling) using cases the agent never saw during iteration.
- Use Case: After launching an email-labeling agent, define a rubric (one label per email, grounded reasons, urgent precision), run up to 5 graded iterations, then validate the passing version against 3 held-back cases before shipping v0.
Quick Start
Run outcome_builder.py against my build sheet with a rubric and 5 max iterations, then use verdict_reader.py on each grader result to decide whether to sharpen, ship, or escalate.
