skill-eval-methodology

What problem does it solve?

Operational discipline for running skill evals. This guide helps prevent data loss, stale artifacts, and wasted iterations by standardizing the evaluation process and ensuring consistent artifact formats and frontmatter.

Core Features & Use Cases

• Pre-flight checklists and guardrails to ensure complete and consistent eval artifacts across runs (evals.json, timing.json, grading.json, benchmark.json, etc).
• Clear guidance on directory structure, artifact generation, and verification steps to enable reproducible evaluations and auditable results.
• Seamless integration with the eval-viewer workflow to compare iterations and track improvements across skill evaluations.

Quick Start

Create an eval iteration directory with evals.json and per-eval folders, then run the validation suite to verify frontmatter and structure.