skill-creator

Start by defining what the skill should do, when it should trigger, and its output format. Then write a SKILL.md with name and description frontmatter plus markdown instructions, create 2-3 realistic test prompts, and iterate based on evaluation results.

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark comparing the two configurations.

Generate 20 realistic eval queries mixing should-trigger and should-not-trigger cases, then run the optimization loop. It evaluates descriptions on a train split, proposes improvements, and selects the best by held-out test score to avoid overfitting.

A good description states both what the skill does and specific contexts when to use it, including phrasings users might actually type. Slightly pushy wording helps because Claude tends to undertrigger skills by default.

Skills with objectively verifiable outputs like file transforms, data extraction, or code generation benefit from quantitative assertions. Skills with subjective outputs like writing style are better evaluated through qualitative human review instead.

On Claude.ai you run test prompts sequentially yourself as a sanity check and collect feedback inline in the conversation. Quantitative benchmarking, blind comparison, and description optimization require subagents or the claude CLI and are skipped.