agentos-api-evals

Run agent performance evaluations by sending requests to API endpoints that execute accuracy, performance, agent-as-judge, and reliability tests. This provides programmatic access to manage agent benchmarking and quality assurance for your agents.

Benchmark agent accuracy before deployment by executing accuracy evaluations against predefined questions and expected outputs. This verifies that agent responses meet your quality standards prior to releasing the agent.

Chain multiple evaluations together to perform comprehensive agent testing. The API supports chaining evals and custom error handling, allowing you to build complex benchmarking workflows across different evaluation types.

Automate accuracy, performance, agent-as-judge, and reliability evaluations for your agents. The API manages these test types alongside listing and filtering capabilities to ensure comprehensive agent benchmarking.

Filter past evaluation runs by accessing API endpoints that support advanced filtering and listing of historical data. Retrieve detailed run data to analyze specific agent performance benchmarks and evaluation outcomes over time.