run-judges

Automated code quality evaluation orchestrates parallel AI judge agents to assess plans and code artifacts, aggregating results into structured JSON reports. You execute specialized judges concurrently to review implementation plans or code for adherence to best practices and test coverage.

LLM evaluation in automated testing uses specialized AI judge agents to review software engineering artifacts against predefined schemas. It runs multiple judges concurrently, validating implementation plans and code outputs to ensure consistent and objective quality assessment.

Configure AI judge evaluation thresholds by using override files to customize assessment strictness. This allows you to adjust automated evaluation criteria for both implementation plans and code artifact types according to your quality requirements.

Yes, you can use AI judges to evaluate software engineering plans before coding starts by running 16 specialized plan evaluation agents concurrently. The judges assess plan quality and validate outputs against predefined schemas into a structured report.

The best way to validate code artifacts against predefined schemas is running parallel AI judge agents that automatically aggregate and validate outputs. This automated testing approach ensures consistent evaluation of code quality and artifact compliance using configurable thresholds.

Automated plan evaluation supports configurable thresholds for different artifact types via override files. You can customize evaluation strictness for both plan and code artifact types, validating outputs against predefined schemas before generating structured JSON reports.