skill-creator

Create, evaluate, and iteratively improve Claude skills with benchmarked test runs.

Updated Mar 31, 2026
One-click install
npx skills add https://github.com/SESELOVSKYDarian/Vase --skill skill-creator-seselovskydarian
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/SESELOVSKYDarian/Vase/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/SESELOVSKYDarian/Vase --skill skill-creator-seselovskydarian

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing an effective skill is hard to get right on the first try, and there is no built-in way to know whether a skill actually improves Claude's output or triggers when it should. This Skill provides a structured draft-test-review-improve loop with quantitative benchmarking so skill quality is measured rather than guessed. ## Core Features & Use Cases - Guided Skill Authoring: Captures intent through interview questions, then drafts SKILL.md files following progressive disclosure and writing-style best practices. - Evaluation & Benchmarking: Runs test prompts with and without the skill in parallel, grades outputs against assertions, and aggregates pass rates, timing, and token usage into benchmark reports with an interactive HTML review viewer. - Description Optimization: Generates realistic trigger/no-trigger query sets and runs an automated optimization loop with train/test splits to improve skill triggering accuracy. - Use Case: A user wants a skill that formats weekly sales reports. This Skill helps draft it, creates test prompts, runs comparison benchmarks, presents results in a browser viewer, and iterates until the outputs satisfy the user. ## Quick Start Help me create a new skill that converts meeting transcripts into structured action-item lists, then test and refine it.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new Claude skill from scratch?

Start by defining what the skill should do, when it should trigger, and its expected output format. Then draft a SKILL.md with name and description frontmatter, create 2-3 realistic test prompts, and iterate based on evaluation results.

How do I test whether my skill actually improves Claude's output?

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both outputs against objective assertions, then aggregate pass rates, timing, and token usage into a benchmark comparison.

How can I improve my skill's description so it triggers correctly?

Generate 20 realistic eval queries mixing should-trigger and should-not-trigger cases, then run the automated optimization loop. It tests description variants against a train/test split and selects the best performer by held-out test score.

What is the difference between with_skill and without_skill benchmark runs?

With_skill runs execute the test prompt with the skill loaded, while without_skill baselines run the identical prompt with no skill. Comparing them isolates the skill's actual contribution to output quality, time, and token usage.

When should I use blind comparison instead of the standard review loop?

Use blind comparison when you need a rigorous judgment between two skill versions, such as verifying a rewrite is genuinely better. An independent agent judges both outputs without knowing which skill produced them, removing bias.

Does skill evaluation work on Claude.ai without subagents?

Yes, but with limitations. Without subagents you run test cases sequentially yourself, skip baseline comparisons and quantitative benchmarking, and present results inline in the conversation for qualitative feedback instead of using the browser viewer.