skill-creator

Create and iteratively improve Claude Code skills with SKILL.md drafts and eval runs.

Updated Jul 11, 2022

Search Tags:#skill creation #benchmarking #evaluation #prompt testing #trigger optimization #iteration loop

Ready to run

npx skills add https://github.com/zhengfran/dotconfig --skill skill-creator-zhengfran

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/zhengfran/dotconfig/tree/main/tools/ai/agents/skills/skill-creator
Command: npx skills add https://github.com/zhengfran/dotconfig --skill skill-creator-zhengfran

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires python3, claude, lsof, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill helps you reliably create new Claude skills or improve existing ones by turning vague ideas into working, testable skill behavior with measurable performance.

Core Features & Use Cases

Skill creation from scratch: turns a user’s goal into a draft skill, then iterates using a test-and-evaluate loop.
Skill improvement and optimization: revises an existing skill based on evaluation feedback, then optionally optimizes description triggering accuracy.
Quantitative and qualitative evaluation: generates evals, runs prompts with and without the skill, grades results, and aggregates benchmark statistics.
Workflow for iterative refinement: supports repeated cycles until the skill is good enough for larger-scale use.

Quick Start

Use the skill creator when you want to “turn this workflow into a Claude skill” and you’re ready to draft a SKILL.md, create a small test set, and iterate based on evaluation results.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I create and benchmark Claude skills with measurable performance?▼

To create and benchmark Claude skills, draft a SKILL.md, generate evaluation prompts, execute baseline and with-skill runs, then grade outputs to aggregate benchmark statistics for measurable performance differences.

What's the best way to improve trigger accuracy for an existing Claude skill?▼

Improving trigger accuracy involves revising the existing skill based on evaluation feedback, then optionally optimizing the description using trigger-eval sets to achieve higher triggering accuracy and repeatable behavior.

How does prompt testing work for evaluating Claude skill behavior?▼

Prompt testing works by running prompts with and without the skill, grading the results based on structured expectations and assertions, and aggregating benchmark statistics to evaluate reliable performance comparisons.

Do I need Python3 and Claude to run skill evaluation and iteration loops?▼

Yes, you need Python3 and Claude dependencies to execute the evaluation runs, grade outputs, and perform the iteration loop required for reliable skill benchmarking and performance comparisons.

When should I use an iteration loop for skill creation instead of manual drafting?▼

Use an iteration loop for skill creation when you need repeatable skill behavior across multiple test cases, requiring structured eval setup to repeatedly cycle through drafts and evaluations until the skill is good enough.

Can I benchmark multiple test cases to optimize Claude skill triggering?▼

Yes, you can benchmark multiple test cases by applying trigger-eval sets and aggregating benchmark statistics, allowing you to quantify performance differences and optionally optimize description triggering accuracy.