evaluation

You evaluate agent performance by feeding outputs to an integrated evaluation runner that applies multi-dimensional rubrics like factual_accuracy, completeness, and tool_efficiency. This produces per-dimension scores, a weighted overall score, and a pass/fail verdict.

Rubric-based evaluation measures non-deterministic agent outputs across multiple dimensions like factual_accuracy, citation_accuracy, and source_quality. It applies weighted scoring to quantify performance, replacing single metrics with a structured approach to track improvements over time.

Yes, you can use tool-call accounting alongside ground-truth comparison for production monitoring. By evaluating agent outputs against rubrics over time, you can detect regressions and track quality during model upgrades or configuration changes.

Yes, the evaluation framework supports ground-truth comparison for testing pipelines and regression checks. It quantifies agent performance by comparing outputs against expected results, generating per-dimension scores and pass/fail decisions to compare configurations.

The best way to quantify agent quality across configurations is applying multi-dimensional rubrics to agent outputs. The runner produces weighted per-dimension scores and an overall pass/fail decision, allowing you to track quality and compare different model setups.

You should avoid using multi-dimensional rubrics when single metrics suffice for your outputs. If your agent produces deterministic, single-path outputs that do not require tracking factual_accuracy, source_quality, or tool_efficiency, a structured rubric approach may be unnecessary.