eval-analyze

To auto-generate an eval.yaml configuration, invoke the eval-analyze skill against your target skill. It recursively reads SKILL.md, sub-skills, prompts, and scripts to derive a grounded evaluation configuration without placeholders.

The best way to analyze skill datasets and derive evaluation thresholds is using recursive skill analysis. It reads your target skill's test cases and files to ensure dataset schema, outputs, and judges are anchored in observed data.

Eval.yaml generation works by deeply reading your target skill directory, including sub-skills and scripts. It produces the exact configuration required by the /eval-run harness, caching an eval.md file for future reference.

Yes, you can use eval-analyze to prepare evaluation infrastructure when eval.yaml is missing. It serves as a trigger integration that automatically analyzes your skill files to output the needed configuration.

You do not need to manually define dataset schema and judges. The tool ensures every field, including models and thresholds, is automatically grounded in the actual files observed in your skill directory.

A key limitation is that configuration generation depends on reading actual files in the skill directory. If prompts, scripts, and test cases are missing or unstructured, it cannot ground the dataset schema and thresholds accurately.