spec-driven-eval

Scores spec-driven implementations against PRD acceptance criteria with binary evidence-backed checklists.

Updated Sep 15, 2026
One-click install
npx skills add https://github.com/Peterson-Benhame/agent-skills --skill spec-driven-eval-peterson-benhame
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: spec-driven-eval
Source: https://github.com/Peterson-Benhame/agent-skills/tree/main/packages/skills-catalog/skills/%28development%29/spec-driven-eval
Command: npx skills add https://github.com/Peterson-Benhame/agent-skills --skill spec-driven-eval-peterson-benhame

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Evaluating whether a spec-driven development (SDD) implementation actually fulfills its PRD is subjective and irreproducible across evaluators. This Skill decomposes every acceptance criterion into atomic binary checks backed by file:line evidence, producing a single comparable final grade that stays stable across runs, evaluators, and frameworks. ## Core Features & Use Cases - Binary checklist scoring: Decomposes each acceptance criterion into MET/UNMET implementation (I) and test (T) checks with mandatory file:line evidence, then rolls up to AC, story, and final grades weighted by P0/P1/P2 priority. - Framework vs harness separation: Grades requirement extraction (Elicitation E), scope adherence (S), robustness (R), engineering gates (G), and test distribution (D) as independent dimensions reported beside the final grade. - Reproducibility controls: Frozen AC baselines, k=3 self-consistency voting, script-computed roll-ups, and calibration anchors keep the same PRD plus commit yielding the same grade. - Use Case: Benchmark three SDD frameworks against one PRD by freezing the baseline once, running plan/implement/evaluate sessions per framework, and comparing the resulting Final scores. ## Quick Start Ask the agent to run spec-driven-eval to grade the implementation on the current branch against the PRD, reusing the frozen baseline and producing a timestamped evaluation report.

Frequently Asked Questions about spec-driven-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate whether a feature was fully implemented against a PRD?

Run spec-driven-eval with the PRD, implementation code, and tests as inputs. It decomposes each acceptance criterion into binary MET/UNMET checks with file:line evidence, then computes a weighted final grade from implementation and test scores.

How to benchmark multiple spec-driven development frameworks on the same PRD?

Freeze the AC baseline checklist once, then run separate plan, implement, and evaluate sessions per framework using fresh chat sessions. Reusing the identical baseline and evaluator keeps the Final scores comparable across frameworks.

What inputs does spec-driven-eval require to grade an implementation?

It requires the PRD as ground truth, the production code, the unit and e2e tests, and optionally the SDD-derived spec.md and tasks.md artifacts. Without derived specs, the Elicitation and Scope axes report as n/a while implementation and test scoring still run.

Does a test that exercises code but does not assert count as coverage?

No. A test that exercises a behavior without asserting the outcome is scored UNMET for verification checks. For persistence checks, the test must assert the real resulting state, such as a database row, not just a mocked call.

When should I not use spec-driven-eval?

Do not use it for planning or building features, writing PRDs, or general code review unrelated to a spec. It only grades completed implementations against acceptance criteria and must be invoked explicitly by name.

Why does the evaluation forbid modifying the code under review?

The evaluator is strictly read-only because fixing code mid-evaluation contaminates the benchmark and invalidates the diff surface. A failing engineering gate is recorded as failed, the Adjusted Final penalty is applied, and the fix is listed in the report instead.