dw-skill-eval-build

To build a test set from skill analysis, load the analysis results from skill-analysis.yaml and generate Happy Path, Edge Case, Adversarial, and Pressure test cases, writing the final output to testset.yaml.

A complete skill evaluation requires four types of test cases: Happy Path, Edge Case, Adversarial, and Pressure tests. These categories ensure comprehensive coverage of normal operations and boundary conditions.

Generating adversarial and pressure test cases involves analyzing skill-intent, api_references, and coverage data from prior analysis results, then structuring inputs that challenge the skill's operational limits.

Yes, constructing a skill test set requires prerequisites including skill-analysis.yaml, eval/config.yaml, and the referenced evaluation pipeline. These files provide the necessary intent and configuration data for test generation.

The generated testset.yaml file should be stored in the eval/evaluations/{skill-name}/ directory. This location facilitates subsequent validation and refinement steps within the evaluation pipeline.

Skill testset validation fails when prerequisites are missing or incomplete. Ensure skill-analysis.yaml contains accurate coverage data and api_references, and that eval/config.yaml properly references the evaluation pipeline before generating the testset.