add-behavioral-tests

Generate pytest behavioral tests and an EvalHub fixture for agent workflows.

2|Updated May 27, 2026
One-click install
npx skills add https://github.com/red-hat-data-services/agentic-starter-kits-skills --skill add-behavioral-tests
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: add-behavioral-tests
Source: https://github.com/red-hat-data-services/agentic-starter-kits-skills/tree/main/skills/add-behavioral-tests
Command: npx skills add https://github.com/red-hat-data-services/agentic-starter-kits-skills --skill add-behavioral-tests

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Creates a complete, repo-aligned behavioral testing setup for an agent by generating pytest suites and an EvalHub fixture, along with the required verification and documentation steps.

Core Features & Use Cases

  • End-to-end behavioral test workflow: guides the full sequence from Jira scope intake through validation and E2E EvalHub execution.
  • MLflow-backed tool-call scoring expectations: focuses on runner/tracing compatibility and ensures tool_calls come from MLflow traces (not unreliable content heuristics).
  • Strict boundary and change control: prevents modifying the agent under test while allowing test-only artifacts and README updates in scope.
  • Use cases: implementing behavioral tests for a new agent, adding behavioral testing when users mention btest/eval coverage/test harness integration, and ensuring MLflow tracing is present for accurate tool scoring.

Quick Start

Invoke the skill with your agent path and the Jira key when available, for example: run /agentic-starter-kits-skills:add-behavioral-tests <agent_path> [JIRA-KEY]

Frequently Asked Questions about add-behavioral-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add behavioral tests to an agent using pytest?

To add behavioral tests, generate a pytest suite and an EvalHub fixture within your agent repository. This establishes end-to-end behavioral coverage, including runner compatibility verification and golden query design, without modifying the agent code.

How do I score tool calls in MLflow traces for e2e evaluation?

Score tool calls by extracting expectations directly from MLflow traces rather than relying on content heuristics. This approach ensures accurate tracing compatibility and reliable tool-call verification during end-to-end behavioral evaluation.

Can I use EvalHub fixtures to test agent workflows without changing the agent code?

Yes, you can use EvalHub fixtures with strict phase-based guardrails that prevent modifying the agent under test. The workflow allows adding test-only artifacts and updating testing documentation while keeping the core agent code unchanged.

What's the best way to set up an end-to-end test harness for agent evaluation?

The best way is implementing a complete behavioral testing setup using pytest suites and EvalHub fixtures. This workflow guides the sequence from scope intake through validation and on-cluster e2e execution, ensuring MLflow tracing is present.

Why does behavioral testing require MLflow tracing verification?

Behavioral testing requires MLflow tracing verification because tool-call scoring depends on traces rather than unreliable content heuristics. Verifying MLflow enrichment ensures the structural inspection of traces accurately captures tool calls for e2e evaluation.