skill-creator

To create and evaluate Claude skills, draft a SKILL.md with YAML frontmatter, assemble eval sets, run parallel baseline and with-skill evaluations, and iterate based on quantitative feedback. This process turns ad-hoc ideas into testable skill packages.

Benchmarking AI skill performance requires running parallel evals to compare baseline and with-skill outputs, collecting timing and token metrics, and grading outcomes. You can optimize triggering by iterating on the skill description using dedicated improvement scripts.

Writing a SKILL.md file requires YAML frontmatter containing the skill name and description, followed by the skill body. The authoring process involves capturing intent, drafting the frontmatter, and writing instructions to ensure reliable triggering and execution.

You can automate the iteration loop by using built-in scripts to draft test cases, spawn parallel eval runs, generate viewable results, draft assertions, and automatically improve descriptions. This loop facilitates continuous quantitative and qualitative refinement.

To run parallel evaluations for Claude skill benchmarking, you need the pyyaml dependency and local script execution capabilities. The environment spawns baseline and with-skill runs, aggregates benchmark statistics, and outputs viewable grading results.