Validate agent prompts against expected outputs using promptfoo.
npx skills add https://github.com/oscardlfr/AndroidCommonDoc --skill eval-agents-oscardlfr
Please help me install this Agent Skill. Skill: eval-agents Source: https://github.com/oscardlfr/AndroidCommonDoc/tree/main/skills/eval-agents Command: npx skills add https://github.com/oscardlfr/AndroidCommonDoc --skill eval-agents-oscardlfr
This Skill ensures that agent prompts produce the expected outputs, helping you catch regressions after editing agent templates.
Run the eval-agents skill to validate all agent prompts.
High-intent search queries and answers about installing and using this skill.
Validating agent prompts for regressions involves running evaluations using promptfoo to test prompt templates against expected outputs. This skill automates that validation process, reporting pass or fail results per test case to identify issues before committing changes.
Yes, promptfoo evaluation can be applied to CI/CD pipelines. The skill integrates into automated workflows to continuously validate agent prompts, ensuring that template updates do not introduce regressions or unexpected output changes.
The best way to catch prompt regressions post-template update is to run an automated evaluation against the modified agent templates. The skill uses promptfoo to compare outputs against expected results, catching regressions before changes are committed.
Yes, promptfoo is required to validate agent prompts and outputs. The skill relies on promptfoo as its underlying engine to execute test cases against agent templates and report whether the actual outputs match the expected results.
Manual testing workflows for agent prompts work by running the evaluation skill locally to validate templates. It applies promptfoo test cases to the prompts, giving developers immediate pass or fail feedback on expected outputs without requiring a full deployment.
The skill provides basic validation logic for agent prompts in CI/CD, focusing on pass or fail results per test case. It requires predefined expected outputs and does not offer advanced internal reasoning or complex test case generation.
C-RossAutomate LLM prompt evaluation tests with a Go-based harness.
SjdjdiejdrirhdkjejGenerate regression tests from bug reports and detected test frameworks.
daymadeConfigure and run Promptfoo LLM evaluation workflows with YAML configs and CLI commands.
EntityProcessOrchestrate AgentV evaluations by simulating LLM responses without API keys.
