skills.rest
raphaelandrews
raphaelandrewsCOMMUNITY
Explore all skills by this author →
Website𝕏@_andrewsshGitHub

skill-creator

Automate evaluation loops to plan, draft, and refine Claude skills.

1|1|Updated May 15, 2025
Search Tags:#automation#claude#evaluation#iteration#skill#prompt-engineering
One-click install
npx skills add https://github.com/raphaelandrews/fsx --skill skill-creator-raphaelandrews
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/raphaelandrews/fsx/tree/main/.agents/skills/skill-creator
Command: npx skills add https://github.com/raphaelandrews/fsx --skill skill-creator-raphaelandrews

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml.

What problem does it solve?

This Skill helps teams systematically craft, test, and improve Claude skills by providing an end-to-end, repeatable loop that captures user feedback, runs evaluations, benchmarks results, and iterates on skill descriptions and behavior.

Core Features & Use Cases

  • Capture Intent: Define what a skill should enable, how it should trigger, and what outputs are expected.
  • Evaluation Loop: Run prompts, collect quantitative metrics, compare configurations, and identify areas for improvement.
  • Iteration & Packaging: Refine skill descriptions, tests, and examples, then package the skill for deployment.
  • Use Case: A team creates a new skill for X, runs a series of evals, analyzes results, and iterates until triggering accuracy is reliable.

Quick Start

Draft a new skill's SKILL.md, wire up an eval set, and start the run loop to test and iterate.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate Claude prompt evaluation and iteration?▼

Automate Claude prompt evaluation by running an automated loop that tests prompts, collects quantitative metrics, and iterates on skill descriptions to improve triggering accuracy.

What is the best way to benchmark Claude skills for reliable triggering?▼

Benchmarking Claude skills involves running a series of evaluations, comparing configurations, analyzing quantitative results, and refining descriptions until triggering accuracy becomes reliable.

How does an automated evaluation loop improve prompt engineering?▼

An automated evaluation loop improves prompt engineering by capturing intent, running prompts against test sets, comparing metrics, and wiring together scripts to iteratively refine behavior and descriptions.

Can I package and deploy Claude skills after testing descriptions?▼

Yes, you can package and deploy Claude skills after refining skill descriptions, tests, and examples through the evaluation loop to ensure reliable triggering accuracy.

Do I need pyyaml to run skill evaluations and generate reports?▼

Yes, pyyaml is required as a dependency to wire together scripts for running evaluations, improving descriptions, generating reports, and packaging Claude skills.

Related Skills

Explore More Software Engineering Skills→
yu13130122297yu13130122297

scout

Identify uncertainties in research framing, datasets, metrics, and baselines.

Community
Intermediate
Bigme2020Bigme2020

skill-creator

Create, modify, and evaluate AI skills with performance benchmarking.

Community
Advanced
Ankh-StudioAnkh-Studio

batch-evaluation

Process multiple artifacts concurrently to generate comparative analysis and trend reports.

Official
Intermediate
Shockwave2kShockwave2k

skill-creator

Draft SKILL.md files and run evaluation loops to measure trigger accuracy.

Community
Advanced