qa-agent-testing

Regression testing for LLM agents validates behavior consistency after modifications. This Skill provides a structured harness with Must-Ace tasks and refusal edge cases to re-run test suites, track quality baselines, and log results across agent versions, ensuring prompt changes don't degrade performance.

A QA harness for agents includes Must-Ace tasks (10 baseline tests), Refusal Edge Cases (5 safety checks), Output Contracts defining expected response formats, and a 6-dimension scoring rubric (0-3 scale per dimension) for consistent evaluation across test runs.

Edge case validation tests agent robustness against boundary conditions and unsafe prompts. This Skill includes predefined Refusal Edge Cases and templates to design structured test scenarios, measure compliance against a scoring rubric, and establish quality baselines before deployment.

Yes. The Skill provides reusable QA harness templates and a regression log to run identical test suites against multiple agent versions, compare scoring results, and track quality drift over time without rebuilding tests for each iteration.

The 6-dimension scoring rubric evaluates agent responses on a 0-3 scale per dimension. Dimensions assess functional correctness, safety compliance, output format adherence, and other quality signals defined in the rubric templates, enabling quantitative regression analysis.

The Skill includes templates and resources to guide harness setup. While familiarity with test design is helpful, the provided Must-Ace task framework and scoring sheets scaffold the process, allowing teams to establish baselines and run regression tests without building infrastructure from scratch.