skills.rest
homeassistant-ai
homeassistant-aiOFFICIAL
Explore all skills by this author →
GitHub

bat-adhoc

Run BAT tests via tests/uat/run_uat.py to validate MCP tools from an AI agent's perspective.

4.3k|185|Updated Sep 14, 2025
Search Tags:#automation#testing#mcp#bat#pr-validation#uat
One-click install
npx skills add https://github.com/homeassistant-ai/ha-mcp --skill bat-adhoc
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: bat-adhoc
Source: https://github.com/homeassistant-ai/ha-mcp/tree/main/.claude/skills/bat-adhoc
Command: npx skills add https://github.com/homeassistant-ai/ha-mcp --skill bat-adhoc

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Bot acceptance testing validates that MCP tools work correctly from a real AI agent's perspective. You design test scenarios dynamically, run them via tests/uat/run_uat.py, and evaluate results.

Core Features & Use Cases

  • Setup prompts to create any entities or state needed for the test
  • Test prompts to exercise the MCP tools and report outcomes
  • Teardown prompts to clean up created entities after the test
  • Use cases include PR validation, regression detection, and end-to-end integration checks with agent CLIs

Quick Start

Pipe a JSON scenario to the test runner to execute BAT against the desired agent.

Frequently Asked Questions about bat-adhoc

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate MCP tools from an AI agent's perspective?▼

You validate MCP tools by designing scenario prompts, executing them via tests/uat/run_uat.py, and evaluating the per-agent results. This bot acceptance testing ensures your tools meet acceptance criteria from a real AI agent's view.

Can I use bot acceptance testing for PR validation and regression detection?▼

Yes, bot acceptance testing applies directly to PR validation, regression detection, and end-to-end integration verification with Claude and Gemini CLIs. It coordinates scenario prompts to exercise MCP tools and report outcomes for these checks.

How do I structure a UAT scenario to test MCP changes?▼

You structure a UAT scenario using setup prompts to create test entities, test prompts to exercise the MCP tools, and teardown prompts to clean up afterward. You then pipe the JSON scenario to the test runner for execution.

Does BAT testing work with Claude and Gemini CLIs?▼

Yes, BAT testing supports end-to-end integration verification with both Claude and Gemini CLIs. The test runner executes your scenario prompts against the desired agent and produces detailed logs of the outcomes.

What is the best way to clean up test entities after validating MCP tools?▼

The best way to clean up test entities is to include teardown prompts in your UAT scenario. After the test prompts exercise the MCP tools and report outcomes, teardown prompts execute to remove any created entities automatically.

How do I execute UAT scenarios for MCP tools?▼

You execute UAT scenarios by piping a JSON scenario directly to the tests/uat/run_uat.py test runner. This runs BAT against your desired agent and generates per-agent results with detailed logs.

Related Skills

Explore More Software Engineering Skills→
yanko-belovyanko-belov
13

test-isolation

Isolate tests with per-test setup and teardown to prevent shared state.

Community
Basic
JongHyunLee84JongHyunLee84

factory-e2e

Execute end-to-end tests against acceptance criteria and classify failures.

Community
Advanced
cyanheadscyanheads
148

field-test

Run end-to-end field tests of MCP tools against a live HTTP server.

Community
Advanced
endorhqendorhq
6

fp-smoke-test

Execute comprehensive smoke tests on CLI, library, web API, and web UI applications.

Official
Intermediate