llm-eval-design
CommunityStructured eval design for LLM performance.
AuthorSokoliem
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Lays out disciplined processes to design, implement, and validate evaluation schemes for LLMs and agents, turning vague assessment into structured, repeatable tests.
Core Features & Use Cases
- Three-tier evaluation: golden cases, adversarial cases, and rubric-based grading to probe capability, robustness, and scoring stability.
- Grader validation and threshold baselines to ensure meaningful progress signals and detect regressions over time.
- CI-friendly wiring and documentation to enable reproducible experiments, dashboards, and evidence collection.
Quick Start
Design an eval suite for a given LLM capability using golden, adversarial, and rubric cases and validate the grader with human labels.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: llm-eval-design Download link: https://github.com/Sokoliem/ultraprompt/archive/main.zip#llm-eval-design Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 510,000+ vetted skills library on demand.