skill-creator

Start by defining what the skill should do, when it should trigger, and its output format. Then write a SKILL.md with name and description frontmatter plus markdown instructions, create test prompts, and iterate based on evaluation results.

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both outputs against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark comparison.

Generate 20 realistic eval queries mixing should-trigger and should-not-trigger cases, then run the automated optimization loop. It tests description variants against train and held-out queries and selects the best performer by test score.

With-skill runs execute the test prompt with the skill loaded, while baseline runs use no skill for new skills or the previous version for improvements. Comparing them isolates the skill's actual contribution to output quality.

Skills with objectively verifiable outputs like file transforms, data extraction, or code generation benefit from assertions. Skills with subjective outputs like writing style are better evaluated through qualitative human review instead.

On Claude.ai you run test prompts sequentially yourself following the skill's instructions, skipping baseline runs and quantitative benchmarking. Review happens inline in conversation rather than through the browser-based viewer.