skills.rest
kegbi
kegbiCOMMUNITY
Explore all skills by this author →
GitHub

promptfoo-evals

Create or update promptfoo evaluation suites with configs, prompts, tests, and assertions.

1|Updated Mar 15, 2026
Search Tags:#testing#evaluation#prompts#workflows#providers#tests#promptfoo
One-click install
npx skills add https://github.com/kegbi/monica-companion --skill promptfoo-evals
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: promptfoo-evals
Source: https://github.com/kegbi/monica-companion/tree/main/.agents/skills/promptfoo-evals
Command: npx skills add https://github.com/kegbi/monica-companion --skill promptfoo-evals

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill provides a structured approach to creating and maintaining promptfoo evaluation suites, enabling teams to scaffold eval configurations, prompts, tests, and providers to ensure robust prompt evaluation.

Core Features & Use Cases

  • Scaffold and manage eval configurations: promptfooconfig.yaml, prompts, tests, and assertions, plus provider references.
  • Debug regressions and extend coverage by adding new evaluation matrices and scenarios.
  • Centralize guidelines and workflows to standardize prompt evaluation across teams.

Quick Start

Create or update a promptfoo eval suite scaffold to cover prompts, tests, and assertions.

Frequently Asked Questions about promptfoo-evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a prompt evaluation suite for testing LLM regressions?▼

To build a prompt evaluation suite, you need to scaffold a promptfooconfig.yaml file that defines prompts, tests, assertions, and providers to systematically catch regressions across models. This establishes maintainable eval matrices.

What is the best way to structure prompt tests across multiple models?▼

The best way to structure prompt tests is by defining provider references and test matrices within a centralized promptfooconfig.yaml, ensuring standardized evaluation coverage across multiple models and scenarios.

Can I use promptfoo to debug prompt regressions and extend test coverage?▼

Yes, you can use promptfoo to debug regressions and extend coverage by updating your eval configurations with new test scenarios, assertions, and prompts to expand the evaluation matrix.

How do I configure assertions and providers in a promptfooconfig.yaml file?▼

You configure assertions and providers in a promptfooconfig.yaml file by structuring them according to promptfoo guidelines, which centralizes workflows and standardizes prompt evaluation across your team.

Does promptfoo work for teams maintaining large eval matrices for prompts?▼

Yes, promptfoo is designed for teams building and maintaining large eval matrices, providing structured configuration files and centralized guidelines to standardize prompt evaluation workflows across models.

Related Skills

Explore More Software Engineering Skills→
sumithrsumithr
5

regen-eval-baseline

Run promptfoo eval baselines for a named skill and store JSON snapshots.

Community
Intermediate
bejranondabejranonda
27

testing-strategies

Define test coverage targets and organize unit, integration, and end-to-end tests.

Community
Basic
daymadedaymade
1.3k

promptfoo-evaluation

Configure and run Promptfoo LLM evaluation workflows with YAML configs and CLI commands.

Community
Advanced
FlexNetOSFlexNetOS

llm-evaluation

Automates LLM evaluation across prompts, outputs, and safety checks with promptfoo, TruLens, and OpenAI evals.

Community
Advanced