sadd:judge-with-debate

Multi-agent debate for consensus-driven evaluation is coordinated by a meta-judge that generates a rubric specification, followed by three independent judge reports and iterative debate rounds to converge on a final score. This workflow applies to code, design proposals, and documentation.

A meta-judge evaluation specification YAML defines reusable rubrics that guide all independent judges through initial scoring and debate rounds. It ensures evidence-based justification and structured consensus across multiple evaluation perspectives.

To avoid groupthink in multi-agent evaluations, separate the initial independent scoring phase from the debate rounds. This ensures three independent judge reports provide distinct perspectives before converging on a consensus score through iterative debate.

Multi-agent debate evaluation works for design proposals, documentation, and code where objective scoring requires multiple perspectives. The meta-judge generates rubrics to evaluate these artifacts and converge on a weighted final score.

Up to three iterative debate rounds are needed to reach a consensus score. The workflow begins with a Phase 0.5 meta-judge and Phase 1 independent judges, then progresses through debate rounds to produce a weighted final score with a consensus report.

Limitations of consensus-driven multi-agent evaluation include the requirement for a generated evaluation specification YAML, three independent judge reports, and a filesystem-based debate protocol. Without these prerequisites, the evaluation workflow cannot execute or converge on a final score.