Define and run end-to-end evaluations for Claude Code sessions with pass@k metrics.
npx skills add https://github.com/ignidus/iscagent --skill eval-harness-ignidus
Please help me install this Agent Skill. Skill: eval-harness Source: https://github.com/ignidus/iscagent/tree/main/skills/eval-harness Command: npx skills add https://github.com/ignidus/iscagent --skill eval-harness-ignidus
Eval-driven development for AI-assisted coding by providing a repeatable, objective framework to define, execute, and track evaluations across model changes and code tasks.
Create and run your first eval using the provided templates and definitions.
High-intent search queries and answers about installing and using this skill.
It uses structured evaluation templates for capability and regression tests, supporting multi-grader setups including Code-Based, Model-Based, and Human Graders to measure pass@k metrics.
The framework supports capability and regression evals with measurable pass@k and pass^k metrics across code changes and prompts, integrating with CI-like workflows and project audits.
This multi-grader approach allows you to combine deterministic grading with model rubrics and human evaluation to track run history and release-ready evaluations.
It implements storage patterns to track baselines, run history, and release-ready evaluations, enabling repeatable and objective assessment of AI coding capabilities.
Without proper baseline management and eval storage patterns, the framework cannot effectively track run history or determine release-ready evaluations across model changes.
GratefulJinx77Evaluate AI agents using code, model, and human graders.
gugug168Define and run capability and regression evaluations for Claude Code sessions.
rongaredeEvaluate Claude Code sessions with capability and regression evals.
robertzengcnReview code diffs, run adversarial tests, and consult on coding questions via the OpenAI Codex CLI.
