ai-evalcheck-and-golden-evals

To validate AI agent behavior after adding a new tool, add corresponding test cases to the evaluation suite and update golden data if necessary. Run evalCheck to confirm correct tool integration and verify no existing functionality breaks.

AI evaluation coverage is validated across seven critical evaluation dimensions. These dimensions ensure comprehensive behavior testing by systematically verifying that all aspects of agent performance have adequate test case representation.

To add new test cases for tool selection accuracy, integrate them into the existing evaluation suite targeting that specific dimension. Update the golden data to match expected outcomes, then run evalCheck to verify the suite's health.

Yes, you can run AI evaluation validation offline by using the equivalent command npm run test. This executes the evalCheck process locally to identify regressions and verify the health of the evaluation suite without external dependencies.

An AI evaluation suite fails after updating golden data when the new data does not align with the expected behavior across the seven evaluation dimensions. Run evalCheck to isolate the regression and verify test case accuracy.