run-evals

Automated regression testing for agent skills is orchestrated by spawning isolated subagents to execute test cases against defined rubrics, generating git diffs and summary reports to verify updates introduce no unexpected behaviors.

Benchmarking agentic behavior performance is achieved through comparative A/B testing, measuring baseline agent performance against versions with specific skills enabled to quantify performance improvements using rubric-based grading.

Yes, you can use subagents to run isolated integration tests for skill evaluations. The orchestration framework manages workspace isolation, ensuring test cases are executed independently against defined rubrics for accurate grading.

Yes, an agent orchestration framework is required to execute skill evaluations. It provides the necessary infrastructure for workspace isolation, git diff generation, and rubric-based grading of the execution outputs.

Automated benchmarking for quality assurance should be used when validating that a new skill update passes all quality criteria, or when you need to compare specific skill performance against a baseline agent to prevent regressions.

Skill evaluation requires workspace isolation to ensure that subagents execute test cases independently without interference. This separation guarantees accurate rubric-based grading and reliable git diff generation for comparative analysis.