judge

Evaluate subagent outputs in multi-agent systems and generate YAML verdict headers.

2|Updated Jul 13, 2025

Search Tags:#evaluation #subagent #meta-judge #structured verdict #debate mode

Ready to run

npx skills add https://github.com/krzemienski/shannon --skill judge-krzemienski

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: judge
Source: https://github.com/krzemienski/shannon/tree/main/skills/judge
Command: npx skills add https://github.com/krzemienski/shannon --skill judge-krzemienski

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

The judge Skill automates the evaluation of subagent outputs, ensuring precision and consistency across various assessment criteria.

Core Features & Use Cases

Automated Evaluation: Evaluate subagent outputs across all /shannon:dispatch* commands.
Structured Verdicts: Generate structured YAML verdict headers for easy parsing by orchestrators.
Two-Stage Pipeline: Utilizes a meta-judge to generate rubric YAML before applying the evaluation.
Iron Rule Validation: Ensures adherence to iron rules like no mocks or test files.
Debate Mode: Handles ties or stakes through multi-round debate with independent judges.
Use Case: When you need to evaluate the outputs of subagents in a multi-agent system, such as in Claude Code plugins.

Quick Start

Run the judge skill on the output of the 'dispatch-parallel' command.

Frequently Asked Questions about judge

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I evaluate subagent outputs in a multi-agent system?▼

Evaluating subagent outputs in a multi-agent system is automated by generating structured YAML verdict headers through a two-stage pipeline, ensuring precise and consistent assessment across various criteria.

How does a meta-judge generate evaluation rubrics for Claude Code plugins?▼

A meta-judge generates evaluation rubrics for Claude Code plugins by creating rubric YAML in the first stage of the pipeline, which is then applied to the subagent outputs to produce structured verdicts.

Can I handle evaluation ties or high-stakes assessments through debate?▼

Handling evaluation ties or high-stakes assessments is supported through a debate mode that triggers multi-round debates with independent judges to reach a final structured verdict.

What are iron rules for validating subagent outputs?▼

Iron rules for validating subagent outputs are strict constraints, such as prohibiting mocks or test files, that the evaluation pipeline enforces to ensure adherence to quality standards.

Does the judge skill work with parallel dispatch commands?▼

The judge skill works with parallel dispatch commands, specifically evaluating the outputs generated across all `/shannon:dispatch*` commands to provide automated assessment and structured verdicts.