skill-creator

Start by defining what the skill should do and when it should trigger, then write a SKILL.md with name and description frontmatter plus markdown instructions. Test it with realistic prompts, gather feedback, and iterate until the outputs are consistently good.

Run each test prompt twice in parallel — once with the skill and once without as a baseline — then grade outputs against objective assertions. Aggregate pass rates, timing, and token usage into a benchmark to compare configurations with mean and standard deviation.

Create about 20 realistic should-trigger and should-not-trigger queries, then run the automated optimization loop. It evaluates the description on a train/test split, proposes rewrites, and selects the best version by held-out test score to avoid overfitting.

The description should state both what the skill does and specific contexts for when to use it, since it is the primary triggering mechanism. Slightly assertive phrasing helps because models tend to undertrigger skills when descriptions are too passive.

Add scripts when test runs show repeated identical helper code being written across cases, and add references when the SKILL.md body approaches 500 lines. This keeps the main instructions lean while making deterministic tasks reusable.

Yes, on Claude.ai you run test prompts sequentially yourself and present outputs inline for feedback, skipping baselines and quantitative benchmarking. The packaging script still works anywhere with Python and a filesystem.