Standardize Claude Code session evaluations with YAML-based repeatable eval artifacts.
npx skills add https://github.com/gganbukim1/myskills --skill eval-harness-gganbukim1
Please help me install this Agent Skill. Skill: eval-harness Source: https://github.com/gganbukim1/myskills/tree/main/eval-harness Command: npx skills add https://github.com/gganbukim1/myskills --skill eval-harness-gganbukim1
Eval-harness helps teams turn AI agent behavior into measurable, repeatable pass/fail outcomes so regressions are caught early and reliability improves over time.
Use eval-harness to define a new feature’s capability evals and required success criteria, then run and report the results to confirm readiness for review.
High-intent search queries and answers about installing and using this skill.
To measure AI agent reliability, you can apply pass@k and pass^k reporting metrics to evaluate success rates and stability across iterative agent and prompt development cycles using structured eval reports.
Eval-driven development (EDD) is a workflow where you define expected behavior before coding, evaluate continuously, and track regressions per change. It applies to LLM-as-judge workflows by using rubric-based model graders to evaluate agent behavior.
Grader types for evaluating Claude Code sessions include deterministic code-based graders, rubric-based model graders using LLM-as-judge, and human review flags designed for ambiguous or high-risk cases.
You set up capability and regression eval templates by defining YAML frontmatter metadata specifying expected behaviors, then running these artifacts to measure new functionality or compare against prior baseline checkpoints.
Yes, you can use human review flags alongside automated benchmarks and model graders to handle ambiguous or high-risk cases during agent reliability measurement, ensuring robust evaluation coverage.
llmh333Define and run Gemini CLI capability and regression evals with pass@k metrics.
sayasaya8039Define pass/fail criteria and run regression evals for Claude Code agents.
GGEduSet up capability and regression evals for Claude Code sessions.
flatrickDefine, measure, and track AI behavior with pass@k metrics.
