create-skill-test

To scaffold eval.yaml files for agent skill tests, generate the required tests directory structure with structured scenarios containing prompts, assertions, rubric items, and timeouts to match repository conventions.

The correct directory structure for skill evaluation tests is tests/<plugin>/<skill-name>/eval.yaml, which organizes scenarios and fixtures safely so that skill evaluations run reliably.

To avoid overfitting when writing rubrics and prompts for eval.yaml scenarios, define robust scenarios with varied assertions and rubric items that test general skill behavior rather than specific expected outputs.

You can set up test fixtures either by copying directories or using inline fixture-based file setup, ensuring that eval.yaml scenarios run reliably with the necessary test data in place.

Required fields when building scenarios in an eval.yaml file include prompt, assertions, rubric, and timeout, with optional declarations for additional required skills or agents needed for the evaluation.

Yes, this skill eval scaffolding approach works for extending existing tests in the dotnet repository by validating target locations and producing correctly structured evaluation scenarios that satisfy established conventions.