write-judge-prompt

To evaluate subjective AI output quality, design an LLM-as-Judge evaluator prompt with clear criteria definitions, few-shot examples, and structured pass/fail verdicts for specific failure modes.

An LLM-as-Judge prompt is a structured evaluator that uses criteria definitions and human-labeled few-shot examples to produce machine-readable binary pass/fail judgments for subjective AI testing criteria.

Create an LLM evaluator by defining explicit pass/fail rules for your targeted failure modes, incorporating human-labeled examples, and avoiding vague or overly broad quality assessments in the prompt.

You need few-shot examples in your LLM judge evaluator when assessing subjective criteria like tone or faithfulness, as these human-labeled examples guide the model to produce reliable binary pass/fail verdicts.

Yes, you can use an LLM-as-Judge prompt to check if AI responses match a required communication style by providing explicit pass/fail rules and human-labeled examples in the evaluator design.

The LLM-as-Judge approach is limited when evaluating vague or overly broad quality assessments, requiring focused criteria definitions and targeted failure mode constraints to ensure reliable machine-readable verdict outputs.