judge-trajectory

Lock a process-aware rubric with 4 to 7 dimensions such as plan suitability, tool correctness, and failure handling, then build an evidence ledger from tool calls and intermediate artifacts. Score every candidate against the same rubric so process quality becomes first-class evidence.

Gather both trajectories including tool calls, failure logs, and intermediate artifacts, then score them against one locked rubric. Separate operational failures from quality failures and run an order-robustness check before finalizing the verdict.

Use trajectory judging when process reliability matters, such as agent runs with tool calls, planning, or side effects. For clean outcome-only response comparison where the final answer is the only signal, switch to an outcome-focused judging contract instead.

Treat narrated chain-of-thought as low-trust evidence unless trace data corroborates it. Prefer observable artifacts, tool call records, and verifier-backed signals over self-reported explanations when scoring trajectories.

Thin traces or incomplete artifacts limit confidence, so verdicts should preserve uncertainty rather than overclaim. Process judgments can also be biased by presentation order, which is why an order-robustness check is required before finalizing.