skill-creator

To create Claude skills, draft a SKILL.md file defining the workflow, generate test prompts, and run parallel baseline comparisons to evaluate pass rates and timing metrics. This automates authoring, testing, and iterating on assistant behaviors.

SKILL.md eval orchestration spawns with-skill and baseline runs, captures timing and token metrics, and grades outputs using reusable grader patterns. It aggregates pass rates to surface analyst notes for high-variance or non-discriminating evaluations.

Yes, you can optimize SKILL.md descriptions automatically by running the description optimization loop. This iteratively improves skill triggering reliability and packages the final workflow into a distributable .skill file.

Yes, you need Python and the pyyaml dependency to run the included scripts for Claude skill benchmarks. These scripts handle spawning runs, aggregating token usage, and generating reviewer-readable benchmark reports.

The best way to benchmark Claude skill performance is to spawn parallel with-skill and baseline runs, capture timing and token metrics, grade outputs with reusable patterns, and aggregate results into reviewer-readable benchmark reports.

High variance in Claude skill evaluations often indicates non-discriminating test prompts or inconsistent grader patterns. Aggregating pass rates and timing metrics into benchmark reports surfaces analyst notes to identify and resolve these evaluation issues.