skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarks and description optimization.

1|Updated Nov 25, 2021
One-click install
npx skills add https://github.com/rstagi/dotfile --skill skill-creator-rstagi
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/rstagi/dotfile/tree/main/.claude/skills/skill-creator
Command: npx skills add https://github.com/rstagi/dotfile --skill skill-creator-rstagi

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing an effective skill is hard to get right on the first try: descriptions may fail to trigger, instructions may be ambiguous, and there is no built-in way to measure whether a skill actually improves Claude's output. This Skill provides a structured draft-test-review-iterate workflow with quantitative benchmarking. ## Core Features & Use Cases - Guided Skill Authoring: Interviews the user about intent, trigger contexts, and output formats, then writes a SKILL.md following progressive-disclosure best practices. - Evaluation Harness: Runs test prompts with and without the skill via parallel subagents, grades assertions, aggregates pass rates, timing, and token usage into benchmark.json, and renders an HTML review viewer. - Description Optimization: Generates trigger/no-trigger eval queries, runs an automated optimization loop with train/test splits, and applies the best-performing description. - Use Case: You want a skill that formats weekly sales reports. This Skill helps you draft it, creates realistic test prompts, runs baseline comparisons, shows results in a browser viewer, and iterates until the outputs pass your review. ## Quick Start Help me create a new skill that converts meeting transcripts into structured action-item lists, then test and improve it.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?

Start by describing what the skill should do and when it should trigger. The skill-creator interviews you about intent and output formats, drafts a SKILL.md with proper frontmatter, then generates test prompts to validate the skill through iterative runs.

How do I test whether my skill actually improves Claude's output?

Run test prompts with and without the skill using parallel subagents, then grade results against assertions. The aggregate_benchmark.py script produces benchmark.json with pass rates, timing, and token deltas between configurations.

How do I improve a skill description so it triggers correctly?

Generate 20 realistic trigger and no-trigger eval queries, review them in the HTML template, then run the run_loop optimization script. It tests description variants against train and held-out queries and returns the best-scoring description.

What files and directories does a valid skill need?

A skill requires a SKILL.md file with YAML frontmatter containing name and description fields. Optional directories include scripts/ for executable code, references/ for on-demand documentation, and assets/ for templates and files used in output.

Does skill evaluation work on Claude.ai without subagents?

Yes, but with limitations. Without subagents you run test cases sequentially yourself, skip baseline comparisons and quantitative benchmarking, and present results inline for qualitative feedback instead of using the browser viewer.

Why does my skill fail to trigger on simple queries?

Claude only consults skills for tasks it cannot easily handle alone, so simple one-step queries may not trigger regardless of description quality. Design eval queries that are substantive, multi-step, and specific enough to benefit from skill guidance.