Execute Freeplay test runs evaluating prompt templates and agents against datasets.
npx skills add https://github.com/freeplayai/freeplay-skills --skill run-test-freeplayai
Please help me install this Agent Skill. Skill: run-test Source: https://github.com/freeplayai/freeplay-skills/tree/main/run-test Command: npx skills add https://github.com/freeplayai/freeplay-skills --skill run-test-freeplayai
This Skill addresses the need to run Freeplay test runs, evaluating prompt templates and agents against datasets, ensuring the quality and performance of AI models.
Run a test for the prompt 'example_prompt' against the dataset 'test_dataset'.
High-intent search queries and answers about installing and using this skill.
To run a test for a prompt template, you execute a Freeplay test run against your specified dataset. This requires explicit user confirmation to proceed, ensuring you avoid unnecessary costs from API calls to LLM providers.
Dataset evaluation for AI agents is the process of testing prompt templates and agents against a collection of data to ensure their accuracy and performance. It utilizes API calls to LLM providers and Freeplay to measure quality.
Yes, you can test new prompts while managing API costs because the test run mechanism requires explicit user consent before execution. This prevents unintended charges from LLM provider and Freeplay API calls.
Prompt testing works with external LLM providers by utilizing API calls to them during the test execution. It integrates with Freeplay to evaluate your prompt templates and agents against your datasets.
You should use a Freeplay test run when you want to test a new prompt or agent against a dataset to ensure its accuracy and performance. It is designed for evaluating prompt templates before deployment.
freeplayaiCreate and manage Freeplay prompt and agent datasets via Python requests.
tiankong0101-byteMeasure system performance and capability differences across iterations.
Strategic-AutomationTrack ML experiments with automatic logging across runs and projects.
copilot-ldEvaluate RAG agent performance with LLM-as-judge criteria, recall, and trace analyses.
