evaluate-cortex-agent

To benchmark a Cortex Agent in Snowflake, you select metrics like correctness and logical_consistency, supply an evaluation dataset, run the evaluation, and review the performance results in Snowsight.

Cortex Agent evaluation metrics include correctness, tool_selection_accuracy, tool_execution_accuracy, and logical_consistency to objectively measure and compare agent performance across configurations.

Yes, you can compare Cortex Agent performance after prompt or tool changes by running scenario-based evaluations to measure improvements and benchmark configurations against previous results in Snowsight.

Yes, you need to prepare or supply a dataset to evaluate a Cortex Agent, which is necessary to measure metrics like correctness and tool_selection_accuracy during the evaluation run.

The best way to track logical consistency in AI agents is using Snowflake native Agent Evaluations to benchmark performance, which requires selecting the logical_consistency metric and running an evaluation with a prepared dataset.