Define evaluation goals, rubrics, and CI/CD pipelines for LLM systems.
npx skills add https://github.com/xcrrr/claude-skills --skill eval-designer
Please help me install this Agent Skill. Skill: eval-designer Source: https://github.com/xcrrr/claude-skills/tree/main/skills/ai-ml/eval-designer Command: npx skills add https://github.com/xcrrr/claude-skills --skill eval-designer
Eval Designer enables teams to design and implement robust evaluation frameworks for LLM systems to measure quality, safety, accuracy, and alignment across prompts, models, and deployments.
Define an evaluation brief for a new LLM feature and generate an accompanying rubric and automated test plan.
High-intent search queries and answers about installing and using this skill.
Design an LLM evaluation rubric by defining scope for safety and alignment, generating test case categories, and establishing thresholds calibrated with human labels to measure system quality.
Automate LLM evaluation pipelines in CI/CD by building test suites and rubrics that run automatically across model changes, capturing versioned run metadata to ensure reproducible safety and regression testing results.
Calibrate LLM evaluation thresholds with human labels by documenting versioned run metadata and comparing automated rubric scores against human-labeled test cases to ensure alignment and measurement accuracy.
You can use automated evaluation pipelines for prompt regression testing by applying defined rubrics and test case categories to CI/CD workflows, measuring safety and accuracy across prompt changes.
Versioned run metadata is needed for LLM benchmarking to document thresholds, track calibration with human labels, and ensure reproducible results across model changes and safety scenarios.
An LLM evaluation brief defines evaluation goals and scope for measuring quality, safety, and accuracy, needed when building automated test plans and rubrics for new LLM features or deployments.
swaylqDiagnose LLM agent production failures by prioritizing evaluation quality over prompt iteration.
wshaddixCompare BenchmarkDotNet results against baselines to detect performance regressions in CI pipelines.
wuyoscarAutomate cross-category toxicity benchmarking for Detoxify classifiers.
simplysmartaiEvaluate LLM applications with automated metrics, human feedback, and LLM-as-judge patterns.
