test-run-analysis

You can analyze Freeplay test run results by using a test run ID to retrieve a summary, surface performance insights, and evaluate metrics like accuracy and format compliance. The Skill handles API requests and returns structured JSON data.

Yes, you can compare two test runs to summarize differences based on various evaluation metrics. This allows you to review performance changes and understand shifts in evaluation scores between runs.

To identify failing test cases, the Skill analyzes test run details and highlights key insights. It surfaces specific test case failures and provides structured performance metrics for your review.

You need API access to Freeplay and environment variables configured for your API key and base URL. The Skill requires the requests library to handle JSON responses containing performance metrics and evaluation scores.

Test case analysis provides evaluation metrics including accuracy, relevance, and format compliance. These metrics are extracted from structured JSON responses to help you understand prompt performance.

You can get a test run summary by providing a specific test run UUID. The summary provides a high-level overview of performance, including metrics and prompt details extracted from the Freeplay API.