Update Evaluation Skill

What problem does it solve?

Manually updating evaluation test cases in Copilot Studio is time-consuming and error-prone, requiring you to switch between local .mcs.yml files and the Copilot Studio interface, validate changes manually, and push updates without guided guardrails to prevent mistakes.

Core Features & Use Cases

• Guided Test Case Modification: Walk through identifying existing evaluation test sets, editing individual test case inputs or expected outputs, adding new test cases, or removing outdated ones with step-by-step instructions.
• Built-in Safety Guardrails: Mandatory pre-change checkpointing to enable easy reverts, category-level quality validation of all test cases in the affected set, and dry-run previews before pushing changes to the live agent.
• Use Case: If you need to update 15+ test cases for your employee self-service agent's benefits inquiry topic after changing the agent's response logic, this skill automates the entire editing, validation, and push process to ensure all tests are updated correctly without manual errors.

Quick Start

Tell the skill you want to update a specific evaluation test case for your Copilot Studio agent, and it will guide you through the entire editing, validation, and push workflow automatically.