skills.rest
varlabz
varlabzOFFICIAL
Explore all skills by this author →
GitHub

skill-creator

Create, evaluate, and iteratively improve AI agent skills with benchmarks.

Updated Apr 30, 2026
Search Tags:#prompt engineering#benchmarking#evaluation#iteration#skill authoring#agent testing#description optimization
One-click install
npx skills add https://github.com/varlabz/skills --skill skill-creator-varlabz
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/varlabz/skills/tree/main/skill-creator
Command: npx skills add https://github.com/varlabz/skills --skill skill-creator-varlabz

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Writing effective agent skills is hard to get right on the first try, and there is no systematic way to know whether a skill actually improves agent behavior. This Skill provides a complete workflow for drafting skills, running test prompts with and without the skill, grading outputs against assertions, and iterating based on quantitative benchmarks and human feedback. ## Core Features & Use Cases - Skill Drafting and Editing: Guides you through capturing intent, interviewing for edge cases, and writing SKILL.md files with proper frontmatter, progressive disclosure, and bundled scripts, references, and assets. - Evaluation and Benchmarking: Spawns parallel test runs with and without the skill, grades outputs against assertions, aggregates pass rates, timing, and token usage into benchmark.json, and renders an HTML review viewer for human feedback. - Description Optimization: Generates realistic trigger/no-trigger eval queries, runs an automated optimization loop with train/test splits, and rewrites the skill description for better triggering accuracy. - Use Case: You want to build a skill that formats weekly sales reports. Use this Skill to draft it, run three realistic test prompts, compare outputs against a no-skill baseline in the review viewer, and iterate until the results consistently pass your assertions. ## Quick Start Help me create a new skill that converts meeting transcripts into structured action-item lists, then test it with a few example prompts.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new AI agent skill from scratch?▼

Start by defining what the skill should do, when it should trigger, and its expected output format. Then write a SKILL.md with name and description frontmatter, draft 2-3 realistic test prompts, and run them with and without the skill to compare results.

How do I test whether my skill actually improves agent output?▼

Run each test prompt twice in parallel: once with the skill and once without as a baseline. Grade both outputs against objective assertions, aggregate pass rates and timing into a benchmark, and review the results in the HTML eval viewer.

How do I optimize a skill description for better triggering?▼

Generate about 20 realistic should-trigger and should-not-trigger queries, review them with the user, then run the automated optimization loop. It evaluates description variants on train and held-out test splits and returns the best-performing description.

What is the difference between with_skill and without_skill benchmark runs?▼

With_skill runs execute the test prompt with the skill loaded, while without_skill baselines run the identical prompt with no skill. Comparing pass rates, time, and tokens between them shows whether the skill adds measurable value.

When should a skill include test cases and assertions?▼

Skills with objectively verifiable outputs like file transforms, data extraction, or code generation benefit from assertions. Skills with subjective outputs like writing style or design are better evaluated qualitatively through human review instead.

Can this skill run in headless environments without a browser?▼

Yes, use the --static flag with generate_review.py to write a standalone HTML file instead of starting a server. Feedback is downloaded as feedback.json, which you copy into the workspace for the next iteration.

Related Skills

Explore More Software Engineering Skills→
yangsi7yangsi7

Specification Creation

Generate technology-agnostic feature specifications with success criteria and acceptance scenarios.

Community
Intermediate
forgefabrikforgefabrik

skill-creator

Drafts SKILL.md files, runs eval comparisons, and iteratively improves Claude skills.

Community
Advanced
entireioentireio
208

session-to-skill

Convert Entire session history into reusable SKILL.md agent definitions.

Official
Intermediate
khoavu882khoavu882

do-implement

Implements code changes directly from descriptions, review findings, or named tasks without requiring planning artifacts.

Community
Advanced