promptfoo

What problem does it solve?

LLM prompt design and evaluation is often manual and error prone. PromptFoo provides a structured framework to systematically test, compare, and improve prompts, reducing bugs and inconsistencies across projects.

Core Features & Use Cases

• Reference test suites: baseline, edge cases, security, and regression tests to validate prompts across models and configurations.
• Red-teaming and vulnerability scanning: integrated checks to uncover prompts that could lead to unsafe outputs or leakage of sensitive information.
• Self-learning workflow: track improvements over iterations and enable continuous prompt optimization in production.
• MCP server integration: support for Claude MCP and easy integration into existing AI stacks.

Quick Start

Run a full prompt evaluation against your Mastra prompts to identify improvement opportunities.