skills.rest
kollaborai
kollaboraiCOMMUNITY
Explore all skills by this author →
GitHub

skill-creator

Create and iteratively improve AI skills with structured evaluation and benchmarking workflows.

1|Updated May 5, 2026
Search Tags:#testing#skill creation#benchmarking#ai agents#evaluation#iterative improvement#description optimization
One-click install
npx skills add https://github.com/kollaborai/kollab --skill skill-creator-kollaborai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-creator
Source: https://github.com/kollaborai/kollab/tree/main/bundles/skills/skill-creator
Command: npx skills add https://github.com/kollaborai/kollab --skill skill-creator-kollaborai

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires yaml, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Creating effective AI skills requires more than just writing instructions—it demands systematic testing, measurement, and iterative refinement to ensure they trigger correctly and perform reliably across diverse user queries.

Core Features & Use Cases

  • Skill Creation & Iteration: Draft new skills from scratch or improve existing ones through a structured capture-intent, write, test, and refine loop.
  • Quantitative Evaluation: Run evals with baseline comparisons, grade outputs against assertions, and aggregate benchmark statistics with variance analysis.
  • Description Optimization: Automatically optimize skill descriptions for better triggering accuracy using train/test splits and iterative improvement loops.

Quick Start

Use the skill-creator skill to build a new skill for extracting invoice data from PDFs, then run the eval loop to optimize its description for reliable triggering.

Frequently Asked Questions about skill-creator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate and benchmark AI agents during skill creation?▼

You can evaluate AI agents by running baseline comparisons, grading outputs against assertions, and aggregating benchmark statistics with variance analysis. This skill provides Python scripts for quantitative evaluation and blind comparison to measure performance.

What's the best way to optimize skill descriptions for triggering accuracy?▼

The best way to optimize skill descriptions is using train/test splits and iterative improvement loops. This skill automates description loop generation to systematically refine and optimize triggering accuracy.

How do I generate test cases for iterative improvement of AI skills?▼

You can generate test cases for iterative improvement by applying a structured capture-intent, write, test, and refine loop. This workflow generates test cases and performs quantitative grading to refine skills end-to-end.

Do I need Python scripts to run quantitative evaluations on AI agents?▼

Yes, you need Python scripts to run quantitative evaluations. This skill requires Python scripts specifically for eval aggregation, benchmark generation, blind comparison, and description loop automation.

Can I perform blind comparisons between baseline AI skills and new versions?▼

Yes, you can perform blind comparisons between baseline AI skills and new versions. The skill provides benchmark generation and blind comparison workflows to measure quantitative improvements accurately.

Related Skills

Explore More Software Engineering Skills→
XWIlluDeluXWIlluDelu

anthropic-skill-creator

Draft, evaluate, and optimize Claude Code skill instructions and triggers.

Community
Advanced
vobasevobase
59

skill-creator

Create and iteratively improve Claude-compatible skills with automated evaluations.

Official
Advanced
sergeyitalysergeyitaly
1

skill-creator

Create and refine AI agent skills through benchmark-driven iterative evaluation.

Community
Advanced
AidenSbVevoAidenSbVevo
1

skill-creator

Create, evaluate, and optimize custom AI skill units with automated testing.

Community
Advanced