Evaluate skill outputs across prompts, baselines, and versions.
npx skills add https://github.com/notwillk/skills --skill evaluate-skill-quality
Please help me install this Agent Skill. Skill: evaluate-skill-quality Source: https://github.com/notwillk/skills/tree/main/skills/evaluate-skill-quality Command: npx skills add https://github.com/notwillk/skills --skill evaluate-skill-quality
Systematic evaluation of skill outputs across varied prompts and baselines helps ensure reliability, detect edge cases, and drive focused improvements.
Run an initial evaluation comparing the current skill version against a baseline to collect timing, tokens, and grading data for refinement.
High-intent search queries and answers about installing and using this skill.
You can evaluate skill quality by applying a structured framework that measures outputs against baselines, tracking performance across iterations using timing data, token counts, and grading artifacts.
An evals.json file provides the structured eval design required for skill benchmarking, defining the prompts and parameters needed to systematically measure and compare skill quality across versions.
Yes, you can automate grading guidance and reporting pipelines by establishing reproducible workspace layouts, allowing you to systematically score outputs and integrate human feedback for improvements.
Yes, PyYAML is required to set up and configure reproducible eval workspaces, providing the dependency needed to parse the structured eval design files used in skill benchmarking workflows.
The best way to compare a current skill version against a baseline is to run an initial evaluation that collects timing, tokens, and grading data to detect edge cases and drive focused improvements.
raphaelchristiConfigure Harness Evolver, LangSmith, and baseline evaluations for Python projects.
ImaginerLabsCreate and refine AI skill definitions with SKILL.md and evaluation workflows.
ArogyaReddyAutomates LLM output evaluation with metrics, human feedback, and benchmarking.
cloudofgeorgeCollect timing metrics from running pages to detect performance regressions.
