critic-judge-design

What problem does it solve?

Designing reliable Critic, Judge, or Evaluator nodes for LLM pipelines requires clear input/output contracts and a robust reasoning order to ensure consistent, trustworthy judgments.

Core Features & Use Cases

• Define a precise primary job for the Critic (what is being judged, the binary outcome, and required inputs).
• Design the input structure and output schema with reasoning preceding the verdict, and choose between single-pass or multi-stage workflows.
• Implement deterministic routing, loop guards, and explicit handling of common failure modes to improve reliability.
• Apply to evaluation tasks such as answer quality assessment, evidence sufficiency checks, or tool-usage validation across stages of an agent.

Quick Start

Provide a clean Critic/Judge design specification for a given evaluation task and output a structured verdict with reasoning before judgment.