skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarked test runs.

1|Updated Jul 6, 2015
One-click install
npx skills add https://github.com/ksolomon/dotfiles --skill skill-creator-ksolomon
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/ksolomon/dotfiles/tree/main/AI/.claude/plugins/marketplaces/claude-plugins-official/plugins/skill-creator/skills/skill-creator
Command: npx skills add https://github.com/ksolomon/dotfiles --skill skill-creator-ksolomon

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing a skill that triggers reliably and produces good outputs is hard to do by intuition alone. This Skill guides the full lifecycle of skill development: drafting SKILL.md files, running test prompts with and without the skill, grading outputs against assertions, and optimizing the description field for accurate triggering. ## Core Features & Use Cases - Skill Drafting and Iteration: Interview the user, write a SKILL.md draft following progressive disclosure patterns, and refine it through repeated feedback loops. - Benchmarked Evaluation: Run test cases in parallel with-skill and baseline configurations, grade assertions, and aggregate pass rates, timing, and token usage into benchmark reports. - Description Optimization: Generate realistic trigger eval queries, review them in an HTML interface, and run an automated train/test loop to find the best-performing description. - Use Case: You want a skill that formats weekly sales spreadsheets. This Skill helps you draft it, creates test prompts, runs comparisons against a no-skill baseline, shows results in a browser-based review viewer, and rewrites the skill until the outputs pass your review. ## Quick Start Ask the assistant to create a new skill for your workflow and walk through drafting, testing, and improving it with evaluations.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?▼

Start by defining what the skill should do and when it should trigger, then write a SKILL.md with name and description frontmatter plus markdown instructions. This Skill guides the interview, drafting, test case creation, and iterative improvement process.

How do I test whether my skill actually improves outputs?▼

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark comparison.

How can I improve my skill description for better triggering?▼

Generate 20 realistic should-trigger and should-not-trigger queries, then run the automated optimization loop. It evaluates description variants on a 60/40 train/test split and selects the best by held-out test score.

What is the required structure of a valid skill directory?▼

A skill requires a SKILL.md file with YAML frontmatter containing name and description, followed by markdown instructions. Optional scripts, references, and assets directories hold executable code, documentation, and output templates.

Does skill evaluation work on Claude.ai without subagents?▼

Yes, but with adaptations: run test cases sequentially yourself instead of spawning parallel subagents, skip baseline runs and quantitative benchmarking, and present results inline in the conversation for qualitative feedback.

When should a skill not include test cases and assertions?▼

Skills with subjective outputs like writing style or design quality are better evaluated qualitatively through human review. Assertions work best for objectively verifiable outputs such as file transforms, data extraction, and fixed workflow steps.