skills.rest
Xueheng-Li
Xueheng-LiCOMMUNITY
Explore all skills by this author →
GitHub

skill-creator

Create, evaluate, and improve Claude skills with SKILL.md drafting and benchmarking.

4|Updated Jan 30, 2026
Search Tags:#automation#benchmarking#claude#packaging#skill-creation#skill-creator#evals
One-click install
npx skills add https://github.com/Xueheng-Li/ai-tools-installation --skill skill-creator-xueheng-li
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/Xueheng-Li/ai-tools-installation/tree/main/recommended_skills/skill-creator
Command: npx skills add https://github.com/Xueheng-Li/ai-tools-installation --skill skill-creator-xueheng-li

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires PyYAML, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Helps authors and teams create, evaluate, and iteratively improve Claude "skills" (SKILL.md-based tool bundles) so that skills reliably trigger, perform deterministic tasks, and include reproducible evals and benchmarks.

Core Features & Use Cases

  • Skill authoring workflow: Guidance and templates for drafting SKILL.md frontmatter, writing the skill body, and organizing bundled resources (scripts, references, assets).
  • Evaluation & benchmarking: Tools and patterns to write evals/evals.json, spawn with-skill and baseline runs, grade outputs, aggregate benchmark statistics, and surface analyst notes.
  • Iteration & packaging: Utilities to improve descriptions, run automated optimize loops, validate skills, and package distributable .skill archives.
  • Use Case: A developer wants to create a new file-conversion skill: use the skill-creator to draft SKILL.md, produce 2–3 realistic test prompts, run with/without-skill comparisons, analyze grades, and iterate until pass rates are satisfactory.

Quick Start

Ask the assistant to help you draft a new SKILL.md, generate 2–3 test prompts and an evals/evals.json, run the evaluation loop, and propose improvements.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a Claude skill with reproducible evaluations and benchmarks?▼

To create a Claude skill, draft a SKILL.md file with frontmatter, generate an evals/evals.json file containing test prompts, and run evaluation loops comparing with-skill and baseline outputs to grade performance and aggregate benchmark statistics.

What is a SKILL.md file and how does it structure Claude skill bundles?▼

A SKILL.md file defines a Claude skill bundle using YAML frontmatter for metadata and a markdown body for instructions, organizing bundled resources like scripts, references, and assets to ensure deterministic task execution and reliable skill triggering.

How do I evaluate and compare Claude skill performance against a baseline?▼

You evaluate Claude skill performance by spawning with-skill and baseline runs using test prompts from evals/evals.json, grading the outputs, and aggregating benchmark statistics with aggregate_benchmark.py to surface analyst notes and pass rates.

Can I automate the iteration and packaging of Claude skills using Python?▼

Yes, you can automate Claude skill iteration by running improve_description.py and automated optimize loops to validate skills, then packaging the validated skill bundles into distributable .skill archives using Python with PyYAML parsing support.

What do I need to run Claude skill evaluation scripts and benchmark aggregations?▼

You need a Python environment with PyYAML installed to run run_eval.py, improve_description.py, and aggregate_benchmark.py, along with a properly organized workspace containing your SKILL.md and evals/evals.json files.

Why are my Claude skills not triggering reliably during testing?▼

Skills fail to trigger reliably when SKILL.md frontmatter and descriptions are poorly optimized; use improve_description.py to refine metadata, run comparative evals, and iterate until with-skill pass rates exceed baseline performance.

Related Skills

Explore More Software Engineering Skills→
viktorbezdekviktorbezdek
10

plugin-evaluation

Run trigger-evals and output-evals to report plugin activation pass rates.

Community
Advanced
koukokouko
6

skill-creator-advance

Create and iteratively improve AI skills with eval-driven development.

Community
Advanced
pynbj1001pynbj1001
2

skill-creator

Create, iterate, and benchmark SKILL.md skill definitions with eval harnesses.

Community
Advanced
tuliosouza99tuliosouza99

skill-creator

Create and iteratively improve SKILL.md-based agent skills with Python eval scripts.

Community
Advanced