skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarked test runs.

Updated May 26, 2026
One-click install
npx skills add https://github.com/robot-denny/ai-sketchlab --skill skill-creator-robot-denny
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/robot-denny/ai-sketchlab/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/robot-denny/ai-sketchlab --skill skill-creator-robot-denny

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing an effective skill is hard to get right on the first try, and without structured testing there is no way to know whether a skill actually improves Claude's output or triggers when it should. This Skill provides a complete workflow for drafting skills, running controlled evaluations against baselines, collecting human feedback, and optimizing trigger descriptions. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user about intent, edge cases, and output formats, then drafts a SKILL.md following progressive disclosure and writing-style best practices. - Benchmarked Evaluation Loop: Runs test prompts with and without the skill in parallel, grades outputs against assertions, aggregates pass rates, timing, and token usage into benchmark reports, and renders an HTML review viewer for qualitative feedback. - Description Optimization: Generates realistic should-trigger and should-not-trigger queries, then runs an automated train/test optimization loop to improve the skill's triggering accuracy. - Use Case: A user says "I want a skill that turns meeting notes into Jira tickets." This Skill drafts the skill, creates test cases, runs with-skill and baseline comparisons, presents results in a browser viewer, and iterates until the user is satisfied. ## Quick Start Help me create a new skill that converts my weekly status notes into a formatted changelog, then test it and improve it based on my feedback.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?

Start by describing what the skill should do and when it should trigger. The skill-creator interviews you about edge cases and output formats, drafts a SKILL.md with proper frontmatter, then runs test prompts to validate behavior before iterating.

How do I test whether my skill actually improves Claude's output?

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both outputs against assertions, aggregate pass rates and timing into a benchmark.json, and review results in the generated HTML viewer.

How can I improve a skill description so it triggers more reliably?

Generate 20 realistic should-trigger and should-not-trigger queries, then run the automated optimization loop. It evaluates description variants on a train split, selects the best by held-out test score, and updates the SKILL.md frontmatter.

Does skill evaluation work on Claude.ai without subagents?

Yes, but with limitations. Without subagents you run test cases sequentially yourself, skip baseline comparisons and quantitative benchmarking, and collect feedback inline in the conversation instead of using the browser viewer.

What is a blind comparison between two skill versions?

A blind comparison gives outputs from two skill versions to an independent judge agent labeled only A and B, so the winner is chosen purely on output quality. A post-hoc analyzer then examines transcripts to explain why the winner won.

Why does my skill not trigger even when the description matches?

Claude only consults skills for tasks it cannot easily handle alone, so simple one-step queries may never trigger regardless of description quality. Use substantive, multi-step eval queries when testing trigger accuracy.