evaluation

To evaluate agent performance, you build a multi-dimensional rubric that scores factual accuracy, completeness, citation accuracy, source quality, and tool efficiency. This approach requires your agent outputs and test sets to conduct a thorough analysis.

Multi-dimensional evaluation is the process of assessing agent outputs using a rubric with specific quality dimensions. It maps dimensions like factual accuracy and tool efficiency to numeric scores, ensuring quality and identifying issues in agent systems.

You set up continuous evaluation by integrating the evaluation rubric into your development workflow. This monitors production quality continuously, applying performance drivers to optimize evaluation budgets and support model upgrades.

Yes, you can compare different configurations or models by applying a consistent multi-dimensional rubric to their outputs. This assesses factual accuracy, completeness, and tool efficiency across test sets to identify the optimal agent setup.

You need your agent outputs and corresponding test sets to assess factual accuracy and tool efficiency. The evaluation applies a multi-dimensional rubric to map these quality dimensions to numeric scores for analysis.

Use a multi-dimensional rubric when basic agent testing cannot capture nuanced quality issues. It is needed when you must evaluate factual accuracy, citation accuracy, and tool efficiency simultaneously to identify performance drivers and optimize budgets.