Generate eval test cases from an eval suite plan or agent description.
npx skills add https://github.com/microsoft/eval-guide --skill eval-generator
Please help me install this Agent Skill. Skill: eval-generator Source: https://github.com/microsoft/eval-guide/tree/main/skills/eval-generator Command: npx skills add https://github.com/microsoft/eval-guide --skill eval-generator
Generates concrete eval test cases for an agent by using an eval suite plan or a plain-English agent description, enabling structured evaluation across lifecycle stages.
Run /eval-generator with your agent description to generate test cases.
High-intent search queries and answers about installing and using this skill.
To generate test cases from an agent description, provide a clear agent task and success criteria. The tool creates 6-8 test cases covering happy-path, edge, and adversarial scenarios without requiring a formal plan.
Agent evaluation test sets are output as a Copilot Studio test set table, a CSV file for single-response imports, and a docx-style report for human review.
No, an eval suite plan is not required. While plan-based generation uses a scenario table and evaluation methods, you can fallback to generating tests from a plain-English agent description.
Yes, multi-turn evaluation is supported. You can generate conversational eval test cases using either an eval suite plan or a plain-English agent description, though CSV output is limited to single-response mode.
The best way to create adversarial scenarios is using the fallback generation mode, which automatically derives 6-8 tests including adversarial cases from a plain-English agent description and success criteria.
The CSV output format is restricted to single-response imports by design. Multi-turn evaluation scenarios are supported in the Copilot Studio test set table and the docx-style report instead.
microsoftGenerate a structured eval plan from an agent description for MS Learn Stage 1 Define.
microsoftTriage Copilot Studio eval results into structured reports with owners and actions.
humanity4aiProvide a test fixture skill for evaluating skill categorization and processing logic.
benediktmsWrite tests that validate production code without mutating it.
