Convert observed LLM failures into structured regression test cases.
npx skills add https://github.com/Alchemication/zdrowskit --skill llm-evals-alchemication
Please help me install this Agent Skill. Skill: llm-evals Source: https://github.com/Alchemication/zdrowskit/tree/main/.agents/skills/llm-evals Command: npx skills add https://github.com/Alchemication/zdrowskit --skill llm-evals-alchemication
This Skill helps maintainers prevent repeated LLM behavior regressions by turning real user feedback and silent model failures into structured evaluation cases.
Use the llm-evals skill to add an evaluation case for a new LLM behavior regression based on an existing failure trace.
High-intent search queries and answers about installing and using this skill.
Build LLM regression tests by converting observed model failures into structured test cases with provenance tracking, fixture design, and deterministic assertions to prevent repeated behavior regressions.
Compare model performance during prompt changes by running route-aware leaderboard records and model comparison runs to verify previously fixed behaviors remain stable.
You can use LLM-as-judge for AI behavior validation by applying selective LLM-as-judge checks alongside deterministic assertions within your evaluation workflow.
pytest supports evaluating tool-use validation in AI applications by structuring regression cases that verify tool-use behaviors and prompt routing remain stable after changes.
Track provenance for LLM evaluation cases by linking structured test cases directly to real user feedback and silent model failures through feedback-driven eval design.
Avoid using deterministic assertions for LLM evaluations when analyzing subjective outputs, where optional judge evaluation provides more accurate behavior validation.
understudylabsGenerate harness, metric, splits, and baseline artifacts from workloads.
SjdjdiejdrirhdkjejGenerate regression tests from bug reports and detected test frameworks.
flext-shStandardize Python testing practices with pytest within the FLEXT framework.
vecearCompare old and new prompt outputs to generate regression reports with diffs and stability metrics.
