skills.rest
adenhq
adenhqCOMMUNITY
Explore all skills by this author →
GitHub

hive-test

Run pytest-based goal-evaluation tests for AI agents and diagnose failures with MCP workflows.

10.9k|5.7k|Updated Jan 12, 2026
Search Tags:#automation#pytest#MCP#evaluation#debug#agents#test
One-click install
npx skills add https://github.com/adenhq/hive --skill hive-test
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hive-test
Source: https://github.com/adenhq/hive/tree/main/.claude/skills/hive-test
Command: npx skills add https://github.com/adenhq/hive --skill hive-test

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill provides a structured framework to validate that AI agents meet their defined goals by running goal-based evaluation tests, surfacing failures, and guiding iterations.

Core Features & Use Cases

  • Verifies that agents meet their goals through repeatable evaluation tests.
  • Provides debugging support with the debug_test workflow to diagnose failures.
  • Enables iterative improvements by re-running tests after modifications to agent code or goals.
  • Integrates with hive-create and MCP tooling to generate test guidelines and orchestrate test runs.

Quick Start

To start, ensure your agent is exported to exports/your_agent and you have a goal_id and goal_json. Use the MCP tools to generate success test guidelines and then write and run tests:

  • List existing tests with mcp__agent-builder__list_tests(goal_id=..., agent_path="exports/your_agent")
  • Generate success criteria test guidelines with the MCP tool
  • Write tests following the guidelines, then run tests with mcp__agent-builder__run_tests(goal_id=..., agent_path="exports/your_agent")
  • If a test fails, diagnose with mcp__agent-builder__debug_test(goal_id=..., test_name="...", agent_path="exports/your_agent")

Frequently Asked Questions about hive-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run goal-based evaluation tests for AI agents?▼

To run goal-based evaluation tests for AI agents, you use the hive-test framework with pytest to verify agents meet their defined goals, then orchestrate runs via the run_tests MCP workflow and report outcomes.

What is goal-based evaluation for AI agents and when do I need it?▼

Goal-based evaluation is a structured testing process that validates AI agents meet their defined goals. You need it to surface failures and guide iterative improvements when refining agent behavior.

How do I debug failing AI agent tests using pytest and MCP?▼

You debug failing AI agent tests by calling the debug_test MCP workflow, which diagnoses failures by passing the goal_id, test_name, and agent_path to isolate issues in agent code or goals.

Can I automate iterative test runs after modifying AI agent code?▼

Yes, you can automate iterative test runs by re-running the mcp__agent-builder__run_tests command after modifications to agent code or goals, enabling continuous validation and improvement cycles.

Do I need a specific directory structure to evaluate AI agents with pytest?▼

Yes, you need to export your agent to an exports/your_agent directory and have a goal_id and goal_json available before generating success test guidelines and running MCP evaluation tests.

Related Skills

Explore More Software Engineering Skills→
edwardmonteiroedwardmonteiro

definition.tech_spike

Creates structured tech-spike charters with evaluation criteria and work plans.

Community
Basic
SinShine5SinShine5
40

testing-agent

Generate, write, and run goal-based evaluation tests for agents using MCP tools.

Community
Advanced
nk10nikhilnk10nikhil

hive

Orchestrate agent development workflows across six hive sub-skills.

Community
Advanced
aleonsaaleonsa

python-testing

Implement pytest-based testing strategies for Python code with TDD, fixtures, and mocking.

Community
Advanced