evaluation

Multi-dimensional rubric evaluation scores agent outputs across dimensions like factual_accuracy, completeness, and tool_efficiency. You provide the agent output and ground truth to generate per-dimension scores, an overall score, pass/fail status, and detailed failure reports.

Multi-rubric scoring quantifies non-deterministic agent behavior by applying configurable dimensions such as factual_accuracy, citation_accuracy, and source_quality. It produces per-dimension scores and an overall assessment to capture real-world performance where single-dimension tests fail.

Test-set management automates standardized evaluations across versions and configurations using TestSet and EvaluationRunner components. You provide agent outputs and corresponding ground truths to run consistent evaluations and generate multi-dimensional performance reports.

Production monitoring samples real interactions to detect quality drift and trigger alerts using the ProductionMonitor component. It applies multi-dimensional rubrics to quantify ongoing agent behavior and maintain performance standards in deployment pipelines.

A configurable evaluation rubric includes dimensions such as factual_accuracy, completeness, citation_accuracy, source_quality, and tool_efficiency. Each dimension receives individual scores alongside an overall score, pass/fail status, and detailed failure reports.

Traditional single-dimension tests fail because agent decisions are non-deterministic and multiple valid outcomes exist. Multi-dimensional rubric evaluation solves this by scoring across configurable dimensions like factual_accuracy and completeness to quantify real-world performance comprehensively.