skills.rest
carrot-hu23
carrot-hu23COMMUNITY
Explore all skills by this author →
GitHub

skill-creator

Draft, evaluate, and package Claude skills with benchmark.json and HTML reports.

905|126|Updated Jan 27, 2023
Search Tags:#automation#benchmarking#skill-development#skill-creator#evals#description-optimization
One-click install
npx skills add https://github.com/carrot-hu23/dst-admin-go --skill skill-creator-carrot-hu23
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/carrot-hu23/dst-admin-go/tree/main/.claude/skills/skill-creator
Command: npx skills add https://github.com/carrot-hu23/dst-admin-go --skill skill-creator-carrot-hu23

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires anthropic, PyYAML, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Reduces the friction of designing, testing, and iterating Claude skills by providing a reproducible workflow for drafting SKILL.md files, generating test prompts, running trigger and baseline evaluations, grading outputs, and producing human-readable benchmark reports.

Core Features & Use Cases

  • Skill authoring: Guided process for drafting SKILL.md frontmatter and detailed instructions to make skills discoverable and reliable.
  • Eval & benchmarking: Orchestrates trigger evaluations, baseline comparisons, grading, aggregation into benchmark.json, and HTML review viewers for human feedback.
  • Iteration & optimization: Automates iterative improvement loops including description optimization, blind comparisons, analyzer suggestions, and packaging into distributable .skill files.
  • Use Case: A developer or QA engineer creating a new skill can draft the skill, create representative evals, run parallel with/without-skill tests, review outputs in the generated viewer, and apply improvements until passing criteria are met.

Quick Start

Use the skill-creator to draft a SKILL.md for a new skill, create 2–3 realistic eval prompts, run one iteration of the eval loop, and open the generated review HTML to inspect grading and benchmark results.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate Claude skill evaluation and benchmarking?▼

To automate Claude skill evaluation, you run parallel trigger evaluations, grade outputs, and aggregate results into benchmark.json. This reproducible workflow supports baseline comparisons and iterative improvement loops for reliable skill testing.

What is the process for drafting and optimizing a SKILL.md file?▼

Drafting a SKILL.md file requires generating YAML frontmatter and detailed instructions. Description optimization then automates iterative improvements using blind comparisons and analyzer suggestions to refine triggers and skill metadata.

How do I package AI skills into distributable files?▼

Packaging AI skills into distributable .skill files involves validating YAML configurations and assembling components like scripts, references, and assets. This ensures the skill is correctly structured for deployment and sharing.

Do I need PyYAML and Anthropic APIs to run trigger evaluations?▼

Yes, running trigger evaluations and invoking external model APIs requires the Anthropic dependency for model interactions and PyYAML for validating SKILL.md frontmatter and configuration files throughout the skill development workflow.

What's the best way to review AI skill benchmark results?▼

The best way to review AI skill benchmark results is through generated HTML review viewers that display grading and benchmark data for human feedback. This allows developers to inspect outputs, compare baselines, and apply improvements until passing criteria are met.

Can I run parallel with and without skill tests for baseline comparisons?▼

Yes, you can run parallel with and without skill tests to perform baseline comparisons. The workflow orchestrates these parallel trigger evaluations, grades the outputs, and aggregates the data to measure skill effectiveness accurately.

Related Skills

Explore More Software Engineering Skills→
MJ-AgentLabMJ-AgentLab

render

Render n8n workflow templates into environment-specific JSON files with PyYAML.

Official
Intermediate
fysoul17fysoul17
4

skill-creator

Design, evaluate, and refine Claude skills with a repeatable workflow.

Community
Advanced
diegocamara89diegocamara89
3

skill-creator

Create and iteratively improve AI skills with evaluation workflows.

Community
Advanced
nodnarbnitramnodnarbnitram
15

{{SKILL_NAME}}

Generate standardized multi-file skill skeleton templates with predefined sections and placeholders.

Community
Basic