Create and iteratively improve Claude-style skill definitions and evaluation pipelines.
npx skills add https://github.com/RyoMa99/dotfiles --skill skill-creator-ryoma99
Please help me install this Agent Skill. Skill: skill-creator Source: https://github.com/RyoMa99/dotfiles/tree/main/dot_claude/skills/skill-creator Command: npx skills add https://github.com/RyoMa99/dotfiles --skill skill-creator-ryoma99
This skill automates the end-to-end process of creating, testing, and iteratively improving Claude-style skills so authors don't have to invent evaluation infrastructure or benchmarking tooling from scratch. It captures intent, drafts SKILL.md content, produces test prompts and assertions, runs comparative with-skill vs. baseline evaluations, grades outputs, aggregates benchmarks, and guides revision loops to produce reliable, well-triggering skill descriptions.
Create a new skill by drafting SKILL.md for the desired task, produce 2–3 realistic test prompts, run the evaluation loop to collect runs and grading, review results in the eval viewer, then iterate on the skill based on feedback.
High-intent search queries and answers about installing and using this skill.
To create and evaluate AI skills, draft a SKILL.md file with metadata and triggers, generate test prompts, run baseline versus with-skill evaluations, grade outputs, and iterate. This automated pipeline handles drafting, benchmarking, and packaging.
The best way to benchmark AI skill performance is running parallel with-skill and baseline evaluations, collecting timing and token metrics, spawning graders to produce grading files, and aggregating results into benchmark statistics for blind comparisons.
Yes, you need a Python-capable environment to run the included skill evaluation scripts. The pipeline relies on Python dependencies like anthropic and pyyaml to orchestrate parallel runs, grade outputs, and package validated skills.
Yes, you can generate realistic test prompts and organize them into eval workspaces. The system recommends objective assertions for quantitative benchmarking, runs comparative evaluations, and aggregates grading results for analysis.
To package validated AI skills, the system generates a distributable .skill file after the evaluation and revision loops complete. It also produces an HTML review viewer to collect human feedback before final packaging.
Unreliable AI skill triggers often stem from vague trigger descriptions in SKILL.md. The revision loop refines trigger descriptions by running blind comparisons and post-hoc analysis on benchmark results to improve activation accuracy.
mamd69Automates drafting, testing, benchmarking, and iterating Claude skill SKILL.md workflows with Python and anthropic/pyyaml libraries.
Leap0920Draft SKILL.md content, generate eval queries, and run trigger/evaluation loops.
diegocamara89Create and iteratively improve AI skills with evaluation workflows.
yokobondAdds custom SVG and PNG costumes to Scratch sprites via VM storage API.
