add-behavioral-tests

Name: add-behavioral-tests
Availability: InStock
Author: red-hat-data-services

Generate pytest behavioral tests and an EvalHub fixture for agent workflows.

2|Updated May 27, 2026

Search Tags:#pytest #tool selection #behavioral tests #mlflow tracing #evalhub #golden queries #e2e evaluation

Ready to run

npx skills add https://github.com/red-hat-data-services/agentic-starter-kits-skills --skill add-behavioral-tests

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: add-behavioral-tests
Source: https://github.com/red-hat-data-services/agentic-starter-kits-skills/tree/main/skills/add-behavioral-tests
Command: npx skills add https://github.com/red-hat-data-services/agentic-starter-kits-skills --skill add-behavioral-tests

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Creates a complete, repo-aligned behavioral testing setup for an agent by generating pytest suites and an EvalHub fixture, along with the required verification and documentation steps.

Core Features & Use Cases

End-to-end behavioral test workflow: guides the full sequence from Jira scope intake through validation and E2E EvalHub execution.
MLflow-backed tool-call scoring expectations: focuses on runner/tracing compatibility and ensures tool_calls come from MLflow traces (not unreliable content heuristics).
Strict boundary and change control: prevents modifying the agent under test while allowing test-only artifacts and README updates in scope.
Use cases: implementing behavioral tests for a new agent, adding behavioral testing when users mention btest/eval coverage/test harness integration, and ensuring MLflow tracing is present for accurate tool scoring.

Quick Start

Invoke the skill with your agent path and the Jira key when available, for example: run /agentic-starter-kits-skills:add-behavioral-tests <agent_path> [JIRA-KEY]

Frequently Asked Questions about add-behavioral-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add behavioral tests to an agent using pytest?▼

To add behavioral tests, generate a pytest suite and an EvalHub fixture within your agent repository. This establishes end-to-end behavioral coverage, including runner compatibility verification and golden query design, without modifying the agent code.

How do I score tool calls in MLflow traces for e2e evaluation?▼

Score tool calls by extracting expectations directly from MLflow traces rather than relying on content heuristics. This approach ensures accurate tracing compatibility and reliable tool-call verification during end-to-end behavioral evaluation.

Can I use EvalHub fixtures to test agent workflows without changing the agent code?▼

Yes, you can use EvalHub fixtures with strict phase-based guardrails that prevent modifying the agent under test. The workflow allows adding test-only artifacts and updating testing documentation while keeping the core agent code unchanged.

What's the best way to set up an end-to-end test harness for agent evaluation?▼

The best way is implementing a complete behavioral testing setup using pytest suites and EvalHub fixtures. This workflow guides the sequence from scope intake through validation and on-cluster e2e execution, ensuring MLflow tracing is present.

Why does behavioral testing require MLflow tracing verification?▼

Behavioral testing requires MLflow tracing verification because tool-call scoring depends on traces rather than unreliable content heuristics. Verifying MLflow enrichment ensures the structural inspection of traces accurately captures tool calls for e2e evaluation.