Implement formal evaluations for Claude Code sessions with pass@k metrics.
npx skills add https://github.com/Rx-K8/my-claude-code --skill eval-harness-rx-k8
Please help me install this Agent Skill. Skill: eval-harness Source: https://github.com/Rx-K8/my-claude-code/tree/main/japanese/skills/eval-harness Command: npx skills add https://github.com/Rx-K8/my-claude-code --skill eval-harness-rx-k8
This Skill provides a structured framework for implementing Evaluation-Driven Development (EDD) principles within Claude Code sessions, treating evaluations as the equivalent of unit tests for AI development.
Use the eval-harness skill to define a new evaluation for the 'user-registration' feature.
High-intent search queries and answers about installing and using this skill.
Evaluation-Driven Development (EDD) formalizes AI evaluation by treating evaluations like unit tests. It defines expected behavior before implementation and executes continuous functional and regression checks throughout the development lifecycle.
You can measure AI regression testing reliability using pass@k and pass^k metrics. These metrics track deterministic and probabilistic checks respectively, ensuring AI performance remains stable over time.
Define a new evaluation within the harness to articulate desired outcomes before implementation. Specify grading types using code-based, model-based, or human grading to verify functional compliance for your feature.
Yes, model-based grading is fully supported as an evaluation type. You can combine it with code-based and human grading to execute comprehensive functional and regression evaluations across your AI development sessions.
The best way to catch AI regressions is implementing continuous evaluation throughout the development lifecycle. By defining expected behavior upfront and tracking reliability with pass@k metrics, you catch issues early.
No external dependencies are required to run EDD evaluations. The harness operates independently using its internal scripts and references to execute functional and regression tests with various grading types.
cescrafliAutomate Claude Code session evaluations with code, model, and human graders.
vinayakgImplement eval-driven development with code-based, model-based, and human grading.
i3ringitDetect and fix visual differences between Figma designs and web implementations.
majiayu000Quantify AI-generated code quality using pass@k and pass^k metrics.
