Convert SKILL.md and references into evals.json, grade.py, and running guides.
npx skills add https://github.com/cr330326/AgentSkill --skill skill-eval-writer
Please help me install this Agent Skill. Skill: skill-eval-writer Source: https://github.com/cr330326/AgentSkill/tree/main/validate/skill-eval-writer Command: npx skills add https://github.com/cr330326/AgentSkill --skill skill-eval-writer
This tool solves the challenge of creating standardized evaluation tooling for any Skill by turning a Skill's SKILL.md and its references into a complete evaluation package. It outputs evals.json and a grade.py script, along with run guidelines, enabling automated validation of instruction quality across Skills.
Run the Skill Eval Writer on a Skill directory to generate evals.json, grade.py, and a grading guide.
High-intent search queries and answers about installing and using this skill.
Skill Eval Writer creates automated eval packs by converting SKILL.md and references into evals.json, grade.py, and running guides. It applies to Skill units requiring deterministic grade scripts and evaluative checks across multiple evals.
An automated eval package includes evals.json with five eval cases, a Python 3.9 compatible grade.py script for grading, optional asset directories, and a generated grading.json with a strict JSON schema for scoring.
Your Skill directory must contain a documented SKILL.md and optional reference files. Providing these inputs allows the Skill Eval Writer to extract evaluation workflows and generate deterministic grade scripts and evaluative checks.
After generating evals.json and grade.py, execute the grade.py script to collect Skill outputs, validate against the strict JSON schema, and output scoring results in grading.json for automated evaluation.
Use Skill documentation generators to structure SKILL.md and reference files, providing structured instruction content that the Skill Eval Writer consumes to generate evals.json and grade.py scripts.
Use standalone Python testing frameworks or manual QA scripts to validate Skill outputs, achieving custom evaluation logic and scoring without relying on auto-generated evals.json or grade.py packages.
VitorAndTxrFragment CLAUDE.md and SKILL.md files into lazy-loaded components to reduce startup token load.
exploreomniExecute Omni CLI eval prompts and score generated query JSON against expected results.
pynbj1001Create, iterate, and benchmark SKILL.md skill definitions with eval harnesses.
rjwaltersRegister the Anvil proposal skill for Claude via a lightweight shim.
