Run blind extraction evaluations against fixtures and grade results with a rubric.
npx skills add https://github.com/brianruggieri/skills --skill eval-extract
Please help me install this Agent Skill. Skill: eval-extract Source: https://github.com/brianruggieri/skills/tree/main/handoff/eval Command: npx skills add https://github.com/brianruggieri/skills --skill eval-extract
Blind evaluation of handoff extraction prompts using blinded subagents to eliminate author bias and ensure consistent scoring across fixtures.
Run the blind evaluation workflow to execute extraction prompts against fixtures and generate scorecards.
High-intent search queries and answers about installing and using this skill.
Blind evaluations of handoff extraction prompts run by loading test fixtures, executing extractor and grader subagents per fixture, and generating scorecards. Results aggregate across multiple iterations to expose extraction accuracy and eliminate author bias.
Grading extraction outputs without author bias uses blinded subagents to score results against a defined rubric. The rubric lives in handoff/eval/prompts/grade.md, ensuring consistent scoring across fixtures during the evaluation workflow.
Automated fixture discovery locates test fixtures within the handoff directory, loads them automatically, and passes each fixture to the extractor agent. This automates the evaluation workflow and generates per-fixture extractions for grading.
Handoff extraction evaluation results and scorecards are saved to handoff/tests/output/eval. This directory stores per-fixture extractions and aggregated summaries generated during the blind evaluation workflow.
The extraction evaluation workflow supports iterative refinement by re-running with fresh subagents. Each execution reloads fixtures, runs the extraction, grades results with the rubric, and outputs new scorecards to track prompt improvements.
Automating handoff extraction grading and result aggregation requires no external dependencies. The workflow handles fixture discovery, prompt loading, subagent execution, rubric grading, and output saving entirely within the local environment.
ChenHomEvaluate skill performance against baselines and analyze failure patterns across test cases.
EntityProcessIteratively refine AI prompt files against AgentV evaluation datasets.
antigravityos187-sketchOrchestrate multi-phase refactoring with a 3-tier subagent architecture.
simplysmartaiEvaluate LLM applications with automated metrics, human feedback, and LLM-as-judge patterns.
