agentv-eval-builder

Name: agentv-eval-builder
Availability: InStock
Author: EntityProcess

Create and manage AgentV YAML evaluation files for AI agent performance testing.

15|3|Updated Nov 7, 2025

Search Tags:#testing #llm #ai agents #evaluation #yaml #agentv

Ready to run

npx skills add https://github.com/EntityProcess/agentv --skill agentv-eval-builder

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: agentv-eval-builder
Source: https://github.com/EntityProcess/agentv/tree/main/plugins/agentv-dev/skills/agentv-eval-builder
Command: npx skills add https://github.com/EntityProcess/agentv --skill agentv-eval-builder

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill streamlines the creation and maintenance of AgentV YAML evaluation files, simplifying the process of testing AI agent performance.

Core Features & Use Cases

Eval File Creation: Generate new AgentV evaluation files from scratch.
Test Case Management: Add, modify, or remove test cases within existing eval files.
Evaluator Configuration: Define and configure various assertion types, including code judges, LLM judges, and rubrics.
Use Case: You need to evaluate a new AI agent's ability to summarize documents. Use this Skill to create an eval.yaml file, define test cases with sample documents and expected summaries, and configure an LLM judge to score the agent's output.

Quick Start

Use the agentv-eval-builder skill to create a new evaluation file for testing agent responses to user queries.

Frequently Asked Questions about agentv-eval-builder

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I create YAML evaluation files for testing AI agent performance?▼

You can create YAML evaluation files for AI agent performance testing by declaratively defining test cases, configuring assertion types like code judges and LLM judges, and structuring evaluation suites within an AgentV eval.yaml file.

What assertion types can I configure in an LLM evaluation suite?▼

In an LLM evaluation suite, you can configure assertion types including code judges, LLM judges, and rubrics to score and validate AI agent outputs against expected test scenarios.

How do I add test cases to an existing AI agent YAML evaluation file?▼

You can add, modify, or remove test cases within an existing AI agent YAML evaluation file by declaratively defining new sample inputs and expected outputs alongside your configured assertion parameters.

Can I use YAML to declaratively define evaluation parameters for AI agents?▼

Yes, you can use YAML to declaratively define evaluation parameters and test scenarios for AI agents, allowing you to structure test cases and configure evaluators without imperative code.

What is the best way to structure evaluation suites for document summarization agents?▼

The best way to structure evaluation suites for document summarization agents is to define test cases with sample documents and expected summaries in a YAML file, then configure an LLM judge to automatically score the output.

Do I need any external dependencies to build AgentV evaluation files?▼

No external dependencies are required to build AgentV evaluation files, as the Skill operates independently to generate and manage YAML definitions for test cases and assertion configurations.