What problem does it solve?
Provides a reproducible, data-driven loop to measure per-rule effectiveness of an AI Skill, identify failures, and iterate wording or examples until every rule passes binary evaluation. This eliminates uncertainty about whether a skill's rules are followed, prevents regressions during compression, and documents discriminating vs non-discriminating rules.
Core Features & Use Cases
- Extracts every rule from a SKILL.md as named assertions and tags them by category for coverage tracking.
- Produces a single full-sweep trap prompt that violates all assertions, runs baseline and with-skill executions, and saves iteration artifacts for auditability.
- Uses cross-model grading (separate grader model) to produce strict PASS/FAIL evidence, root-cause failures, discriminating flags per assertion, and benchmark reports.
- Ideal for improving an existing skill, validating a new skill before deployment, compressing a skill without regression, and measuring variance reduction across runs.
Quick Start
Run a full-sweep evaluation: extract assertions, write one trap prompt that violates every rule, run baseline and three with-skill runs, then grade outputs with a separate cross-model grader and record benchmarks.
