Benchmark local AI skills against held-out evaluation prompts with deterministic sampling.
npx skills add https://github.com/LumenWipe/lumenwipe --skill bench-skills
Please help me install this Agent Skill. Skill: bench-skills Source: https://github.com/LumenWipe/lumenwipe/tree/main/.claude/skills/bench-skills Command: npx skills add https://github.com/LumenWipe/lumenwipe --skill bench-skills
This Skill replaces guesswork with reproducible measurements of how well a skill handles representative prompts, helping you determine whether an optimization actually improved quality.
Use the bench-skills skill to benchmark my skills with up to five samples and seed 42.
High-intent search queries and answers about installing and using this skill.
To measure skill quality improvements, you can evaluate your optimized AI skills against held-out evaluation prompts to generate reproducible quality scores and compare them with recent backups.
Detect regressions in AI skills by comparing current versions against recent backups using held-out evaluation sets, fixed judging seeds, and deterministic sampling to report score deltas.
Yes, you can benchmark multiple local skills simultaneously by applying deterministic sampling and fixed judging seeds to compare mean scores and verify which optimizations produced measurable improvements.
Generate evaluation sets for skill benchmarking by building synthetic prompts or extracting relevant local traces to create stable, held-out datasets for reproducible quality measurement.
Yes, skill benchmarking supports optional JSONL result recording, allowing you to log held-out evaluation scores, quality measurements, and regression detection outputs for further analysis.
Ensure reproducible LLM judging by using deterministic sampling, fixed judging seeds, and a strict one-to-five quality scale to evaluate expected responses consistently across benchmarking runs.
marcusgollDetect breaking API, schema, and interface changes with severity levels and migration paths.
devs6186Analyze draft prompts and map them to ECC components for optimized output.
gavin100305Create and iteratively improve Claude skills with SKILL.md and benchmark evals.
Dianshu-LiaoImplement LLM evaluation with automated metrics and LLM-as-judge patterns.
