sadd:judge-with-debate

Multi-agent consensus evaluation is automated by coordinating independent judges in iterative debate rounds. Three separate judge agents analyze solutions against a shared evaluation specification, defending and revising scores up to three rounds until reaching overall score convergence.

A multi-round debate approach for evaluation uses independent judges to produce initial assessments with evidence per criterion. Judges then engage in iterative exchanges to defend, challenge, and revise scores over up to three rounds until a consensus report is generated.

To set up a multi-judge evaluation workflow, a meta-judge first generates a tailored evaluation YAML specification. Three separate judge agents then use this shared specification to perform independent analysis, exchanging filesystem-based reports to debate and reach consensus.

Yes, rubrics and a meta-judge standardize automated evaluation by generating a shared evaluation specification YAML. This tailored rubric is applied across all three independent judges and iterative debate rounds to ensure fair, evidence-driven consensus for any code or documentation solution.

If independent judges cannot reach score convergence during the debate rounds, the process supports escalation to human review. The multi-round debate runs up to three rounds of iterative exchanges where judges defend and challenge scores before this fallback is triggered.

No, automated multi-judge consensus requires no external dependencies or components. The workflow operates entirely through a meta-judge generating evaluation specifications, three independent judge agents, and filesystem-based report exchanges to coordinate the iterative debate rounds.