Implement eval-driven development with code, model, and human graders.
npx skills add https://github.com/Lincyaw/cc-md --skill eval-harness-skill-lincyaw
Please help me install this Agent Skill. Skill: Eval Harness Skill Source: https://github.com/Lincyaw/cc-md/tree/main/skills/eval-harness Command: npx skills add https://github.com/Lincyaw/cc-md --skill eval-harness-skill-lincyaw
This Skill provides a structured framework for evaluating AI model performance, ensuring reliability and tracking regressions through formal evaluation processes.
Use the eval harness skill to define a new capability evaluation for user authentication.
High-intent search queries and answers about installing and using this skill.
Eval-driven development is a practice where evaluations are defined before coding to ensure AI reliability. You need it when developing AI features and want to formally track performance and prevent regressions.
To prevent regressions in AI models, implement regression evaluations to ensure existing functionalities remain intact after code changes. This framework tracks performance using pass@k and pass^k metrics.
You can set up capability evaluations by defining tests for new functionalities the AI should possess. This framework supports code-based, model-based, and human grading for comprehensive assessment.
AI model reliability is tracked using pass@k and pass^k metrics. These metrics measure performance during capability and regression testing to ensure consistency and prevent regressions.
Yes, you can use code-based, model-based, and human grading for AI evaluation. This allows comprehensive assessment by combining automated checks with human oversight to ensure reliability.
No, this evaluation framework does not require external dependencies to run. It provides a standalone structured framework for evaluating AI model performance and tracking regressions.
selcukyucelGenerate structured evaluation datasets with binary rubric scoring for AI outputs.
AndyHsuTWImplement eval-driven development with code-based, model-based, and human grading.
Ankh-StudioRun automated regression tests comparing evaluations against baseline metrics.
beratcelik1Define and run formal evaluations for AI-assisted development tasks.
