eval-extract

Run blind extraction evaluations against fixtures and grade results with a rubric.

Updated Mar 18, 2026

Search Tags:#evaluation #agents #handoff #fixtures #rubric #extraction #scorecards

Ready to run

npx skills add https://github.com/brianruggieri/skills --skill eval-extract

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-extract
Source: https://github.com/brianruggieri/skills/tree/main/handoff/eval
Command: npx skills add https://github.com/brianruggieri/skills --skill eval-extract

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Blind evaluation of handoff extraction prompts using blinded subagents to eliminate author bias and ensure consistent scoring across fixtures.

Core Features & Use Cases

Loads the grading rubric and test fixtures, runs extractor and grader agents per fixture, and outputs per-fixture extractions and scorecards.
Saves results to handoff/tests/output/eval and provides an automated summary across fixtures.
Supports iterative refinement by exposing evaluation workflow that can be re-run with fresh agents.

Quick Start

Run the blind evaluation workflow to execute extraction prompts against fixtures and generate scorecards.

Frequently Asked Questions about eval-extract

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run blind evaluations of handoff extraction prompts against test fixtures?▼

Blind evaluations of handoff extraction prompts run by loading test fixtures, executing extractor and grader subagents per fixture, and generating scorecards. Results aggregate across multiple iterations to expose extraction accuracy and eliminate author bias.

What is the best way to grade extraction outputs using a rubric without author bias?▼

Grading extraction outputs without author bias uses blinded subagents to score results against a defined rubric. The rubric lives in handoff/eval/prompts/grade.md, ensuring consistent scoring across fixtures during the evaluation workflow.

How does automated fixture discovery work for handoff extraction testing?▼

Automated fixture discovery locates test fixtures within the handoff directory, loads them automatically, and passes each fixture to the extractor agent. This automates the evaluation workflow and generates per-fixture extractions for grading.

Where are handoff extraction evaluation results and scorecards saved?▼

Handoff extraction evaluation results and scorecards are saved to handoff/tests/output/eval. This directory stores per-fixture extractions and aggregated summaries generated during the blind evaluation workflow.

Can I re-run the extraction evaluation workflow iteratively to refine prompt performance?▼

The extraction evaluation workflow supports iterative refinement by re-running with fresh subagents. Each execution reloads fixtures, runs the extraction, grades results with the rubric, and outputs new scorecards to track prompt improvements.

Do I need external dependencies to automate handoff extraction grading and result aggregation?▼

Automating handoff extraction grading and result aggregation requires no external dependencies. The workflow handles fixture discovery, prompt loading, subagent execution, rubric grading, and output saving entirely within the local environment.