eval-execution

To run end-to-end tests for AI agent workflows, activate a Python virtual environment, build the project with make ci-build, start a local evaluation server on an assigned port, and execute your E2E test flows against it.

A smoke regression check for AI agents validates workflow behavior locally before PRs. You need it when modifying AI agent workflows to ensure existing functionality remains intact through automated benchmark runs and local end-to-end checks.

Yes, you can execute benchmark runs for repository-focused AI workflows. The Skill automates benchmark execution by starting a local evaluation server and running tests to measure and verify AI agent workflow performance locally.

Yes, you need to build the project before starting the local evaluation server. The process requires activating a Python virtual environment and running make ci-build to compile the project, followed by starting the server on an available port.

You can inspect health and logs during an AI agent workflow evaluation run by using the provided utilities and commands. These tools allow you to monitor the local evaluation server status and review outputs from your E2E tests and benchmark runs.

The best way to validate AI agent workflow changes locally without deploying is to run smoke regression tests and E2E checks. Start a local server on an assigned port after building with make ci-build to verify behavior safely.