skill-evaluator

You can evaluate skill performance against a baseline by creating eval prompts and rubrics, then running a blind comparison to track regression across versions. This identifies how well a skill performs and highlights failure patterns across test cases.

Rubric-based grading for benchmark regression analysis is a process of scoring outputs against predefined criteria to track performance changes across versions. It helps analyze fail clusters, variance, and edge cases to guide improvements.

You build eval prompts and rubrics by defining specific test cases and grading criteria within an evaluation plan. This process measures skill performance by applying deterministic checks and summarizing benchmark results.

Yes, you can run a blind comparison to compare outputs across skills by reading and comparing SKILL.md frontmatter and related evaluation data. This supports validating eval prompts and highlighting variance.

Yes, this approach supports deterministic checks and benchmark summaries to validate skills. It requires reading and comparing SKILL.md frontmatter and related evaluation data to accurately assess performance against a baseline.

You need transcript analysis for failure patterns in test cases when you want to identify how well a skill performs against a baseline. It helps analyze fail clusters and edge cases to guide improvements and track regression.