skill-creator

Automated skill development and benchmarking uses a structured framework to define intent, run quantitative tests, and refine performance for new automation tasks. This Skill provides scripts and references to guide creation, measurement, and optimization workflows.

Creating a skill from scratch involves defining its purpose, structure, and functionality, then testing and refining its performance. The Skill Creator framework guides users through these steps, making skill development accessible even with limited coding experience.

Skill optimization improves triggering accuracy and engagement by refining descriptions and iterating based on quantitative and qualitative evaluation metrics. The framework analyzes performance data to suggest specific structural and descriptive adjustments.

Automated benchmarking evaluates skill performance for data analysis tasks by running the skill against sample inputs and aggregating results. The framework applies robust evaluation mechanisms to measure outputs, facilitating continuous improvement across diverse scenarios.

Running skill evaluation and aggregating benchmark results requires specific dependencies like generate_review.py and aggregate_benchmark.py. These scripts process performance data and generate the metrics needed for continuous skill refinement.

A skill fails to trigger correctly when its description lacks optimization for accurate intent matching. Using the Skill Creator's description optimization features refines the triggering logic, ensuring the skill activates appropriately across diverse scenarios.