skills.rest
spulido99
spulido99COMMUNITY
Explore all skills by this author →
GitHub

Evals-Driven Development

Define scenarios, execute tests, and analyze results for agent evaluations.

2|Updated Jan 15, 2026
Search Tags:#testing#evaluation#EDD#agent evaluation#deep agent systems
One-click install
npx skills add https://github.com/spulido99/claude-toolkit --skill evals-driven-development
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Evals-Driven Development
Source: https://github.com/spulido99/claude-toolkit/tree/main/plugins/deepagents-builder/skills/evals
Command: npx skills add https://github.com/spulido99/claude-toolkit --skill evals-driven-development

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pytest, agentevals, openevals, langsmith, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a structured approach to evaluating and iterating on deep agent systems, ensuring they meet their intended goals with high accuracy and reliability.

Core Features & Use Cases

  • Define Scenarios: Create detailed scenarios from Job-To-Be-Done descriptions, covering happy paths, edge cases, and failures.
  • Evaluate Agents: Use snapshot testing, smoke testing, and full eval reviews to assess agent performance and functionality.
  • Iterate and Expand: Add new scenarios, refine existing ones, and expand the dataset for continuous improvement.
  • Multi-Agent Systems: Evaluate routing, subagent execution, and system-level integration with hierarchical evaluation patterns.
  • Safety and Security: Ensure agents handle edge cases, provide correct responses, and comply with security protocols.
  • Use Case: Imagine you are developing an AI agent for customer support. Use this Skill to design test scenarios, evaluate its performance, and ensure it handles edge cases like account recovery and order tracking.

Quick Start

Run the /design-evals command to generate a baseline eval suite from your JTBD.

Frequently Asked Questions about Evals-Driven Development

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate deep agent systems using test scenarios?▼

Evaluate deep agent systems by defining scenarios from Job-To-Be-Done descriptions, executing tests, and analyzing results. This structured approach covers happy paths, edge cases, and failures to ensure high agent accuracy.

What is evals-driven development for AI agents?▼

Evals-driven development is a structured approach to iterating on AI agents. It uses snapshot testing, smoke testing, and full eval reviews to assess agent performance and expand test datasets for continuous improvement.

How do I test multi-agent system routing and subagent execution?▼

Test multi-agent systems by applying hierarchical evaluation patterns. This evaluates routing, subagent execution, and system-level integration to ensure correct responses and compliance with security protocols.

Does this evaluation workflow work with pytest and langsmith?▼

Yes, the evaluation workflow operates with pytest, agentevals, openevals, and langsmith. You execute tests and analyze agent performance using these libraries alongside your agent definitions and scenario YAML files.

How do I generate a baseline eval suite from a Job-To-Be-Done description?▼

Generate a baseline eval suite by running the design-evals command. This creates initial test scenarios from your Job-To-Be-Done descriptions, covering edge cases and failures for your specific agent workflow.

Related Skills

Explore More Software Engineering Skills→
yu13130122297yu13130122297

scout

Identify uncertainties in research framing, datasets, metrics, and baselines.

Community
Intermediate
jonnathan-lsjonnathan-ls

skill-creator

Creates, tests, and iterates on Claude skills with validation and metrics.

Community
Advanced
dhar174dhar174
1

langgraph-testing-evaluation

Generate test scaffolds and evaluate LangGraph and LangChain agents.

Community
Advanced
jankneumannjankneumann
4

gen-eval-scenario

Create gen-eval scenario YAML files with schema validation.

Community
Advanced