eval-agents

Validate agent prompts against expected outputs using promptfoo.

1|Updated Mar 9, 2026

Search Tags:#CI/CD #validation #regression testing #agent prompts #promptfoo

Ready to run

npx skills add https://github.com/oscardlfr/AndroidCommonDoc --skill eval-agents-oscardlfr

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: eval-agents
Source: https://github.com/oscardlfr/AndroidCommonDoc/tree/main/skills/eval-agents
Command: npx skills add https://github.com/oscardlfr/AndroidCommonDoc --skill eval-agents-oscardlfr

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill ensures that agent prompts produce the expected outputs, helping you catch regressions after editing agent templates.

Core Features & Use Cases

Promptfoo Evaluation: Run evaluations against agent prompt templates using promptfoo.
Output Validation: Validates agent prompts and reports pass/fail results per test case.
Use Case: After updating agent templates, use this Skill to automatically validate the prompts and identify any issues before committing changes.

Quick Start

Run the eval-agents skill to validate all agent prompts.

Frequently Asked Questions about eval-agents

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I validate agent prompts for regressions after updating templates?▼

Validating agent prompts for regressions involves running evaluations using promptfoo to test prompt templates against expected outputs. This skill automates that validation process, reporting pass or fail results per test case to identify issues before committing changes.

Can I run promptfoo evaluation in CI/CD pipelines?▼

Yes, promptfoo evaluation can be applied to CI/CD pipelines. The skill integrates into automated workflows to continuously validate agent prompts, ensuring that template updates do not introduce regressions or unexpected output changes.

What is the best way to catch prompt regressions post-template update?▼

The best way to catch prompt regressions post-template update is to run an automated evaluation against the modified agent templates. The skill uses promptfoo to compare outputs against expected results, catching regressions before changes are committed.

Do I need promptfoo to validate agent prompts and outputs?▼

Yes, promptfoo is required to validate agent prompts and outputs. The skill relies on promptfoo as its underlying engine to execute test cases against agent templates and report whether the actual outputs match the expected results.

How does manual testing workflow for agent prompts work?▼

Manual testing workflows for agent prompts work by running the evaluation skill locally to validate templates. It applies promptfoo test cases to the prompts, giving developers immediate pass or fail feedback on expected outputs without requiring a full deployment.

What limitations exist when validating agent prompts in CI/CD?▼

The skill provides basic validation logic for agent prompts in CI/CD, focusing on pass or fail results per test case. It requires predefined expected outputs and does not offer advanced internal reasoning or complex test case generation.