judge-outcome

Evaluate final outputs and candidate responses against locked rubrics for outcome-based comparison.

Updated Mar 25, 2026
One-click install
npx skills add https://github.com/Tyler-R-Kendrick/copilot-auto-training --skill judge-outcome-tyler-r-kendrick
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: judge-outcome
Source: https://github.com/Tyler-R-Kendrick/copilot-auto-training/tree/main/skills/judge-outcome
Command: npx skills add https://github.com/Tyler-R-Kendrick/copilot-auto-training --skill judge-outcome-tyler-r-kendrick

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve? Comparing candidate outputs or picking the best final answer is error-prone when judgments rely on gut feel, presentation polish, or invented reasoning about the process. This Skill enforces a disciplined, evidence-anchored method for judging end-state quality without needing full execution trajectories. ## Core Features & Use Cases - Locked Outcome Rubrics: Define 3-7 task-specific scoring dimensions with explicit pass/partial/fail boundaries before judging any candidate. - Bias-Aware Comparison: Run order-robustness checks, distrust unsupported chain-of-thought, and calibrate confidence when evidence is thin. - Decision Packages: Produce a concise verdict with the winner, margin, decisive evidence, rejected-candidate failure modes, and uncertainty notes. - Use Case: Given two candidate support replies, a reference answer, and explicit criteria, the Skill locks a shared rubric, scores both replies on end-state evidence, and reports the stronger outcome with a calibrated confidence level. ## Quick Start Compare these two candidate responses using only the final outputs and the provided reference criteria, then pick the stronger outcome with a confidence note.

Frequently Asked Questions about judge-outcome

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare two AI-generated responses and pick the best one?

Lock a shared rubric of 3-7 outcome dimensions before judging, score both candidates against the same criteria using only final-output evidence, and report the winner with margin and confidence. Run an order-robustness check to catch position bias.

What is outcome-based judging versus trajectory-based judging?

Outcome judging evaluates only end-state artifacts like final answers, files, or benchmark results, while trajectory judging examines tool traces, intermediate steps, and side effects. Use outcome judging when the decisive evidence is final answer quality rather than process behavior.

When should I not use outcome-only evaluation?

Avoid outcome-only judging when tool traces, runtime failures, side effects, or intermediate artifacts are central to the verdict. In those cases, switch to a process-aware judging contract that inspects the full execution trajectory.

How do I reduce bias when judging pairwise response comparisons?

Swap the candidate order and check whether the verdict changes; if it does, report reduced confidence instead of hiding the instability. Also treat rhetorical polish and narrated chain-of-thought as low-trust evidence unless final artifacts corroborate them.

What should an outcome judging decision package include?

Include the selected candidate and margin, the locked rubric, a decisive evidence summary, the main failure modes of rejected candidates, and a confidence or uncertainty note. A reusable rubric and evidence-ledger template is provided in the assets directory.