LLM-as-Judge Skill

What problem does it solve?

Automatically assess the quality of agent outputs using configurable criteria to ensure consistent judgments and reduce manual review effort.

Core Features & Use Cases

• LLM-based evaluation of responses across criteria such as relevance, coherence, helpfulness, and factual accuracy.
• Supports tools like judge_output, batch_judge, and get_judge_config to produce scores, feedback, and configuration data.
• Use Case: benchmark and compare different agent responses in customer support, tutoring, or informational assistants to identify strengths and gaps.

Quick Start

Provide a prompt and agent response along with evaluation criteria to receive a structured quality score and actionable feedback.