Automate formal evaluation workflows for Claude Code sessions with defined graders and lifecycle stages.
npx skills add https://github.com/flag3/dotfiles --skill eval-harness
Please help me install this Agent Skill. Skill: eval-harness Source: https://github.com/flag3/dotfiles/tree/main/.claude/skills/eval-harness Command: npx skills add https://github.com/flag3/dotfiles --skill eval-harness
This Skill provides a structured framework to define, execute, and observe evaluations for Claude Code sessions, enabling repeatable, traceable improvements through eval-driven development.
Define a new evaluation workflow for a Claude Code feature and run: /eval define <feature-name>
High-intent search queries and answers about installing and using this skill.
Eval-harness automates formal evaluation workflows by defining capability and regression tests with multiple grader options. Start with /eval define <feature-name> to create a structured workflow, then execute evaluations with defined storage for definitions, logs, and baselines to track improvements systematically.
Capability evals verify new features behave as intended, while regression evals check that existing functionality remains intact after changes. Eval-harness supports both through a single framework, enabling you to test new behavior and prevent breakage in one workflow.
Yes, eval-harness supports Code-Based, Model-Based, and Human graders for flexible assessment within a single evaluation framework. This lets you choose the most appropriate grading method for each test without switching tools.
Eval-driven development enables repeatable, traceable improvements by automating formal evaluations with defined lifecycles from define to report. You gain structured feedback on capability and regression tests, making incremental improvements measurable and observable.
Eval-harness requires frontmatter metadata to define evaluations and provides structured storage for eval definitions, execution logs, and baselines. This enables reproducible workflows and historical tracking of evaluation results across sessions.
deepan-alveDefine and run structured capability and regression evals for Claude Code sessions.
GoBeromsuDefine and run evaluations for Claude Code changes with structured graders and reports.
rambozRun iterative agent loops with guardrails and gate.py scoring.
quanganh208Create or update Claude skills with eval-driven iteration and bundled resources.
