mcp-tool-surface-testing

Tests every tool an MCP server exports using an expectations-first black-box audit.

2|Updated Jul 18, 2026
One-click install
npx skills add https://github.com/Arasz/ai-badger --skill mcp-tool-surface-testing-arasz
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: mcp-tool-surface-testing
Source: https://github.com/Arasz/ai-badger/tree/main/features/mcp/skills/mcp-tool-surface-testing
Command: npx skills add https://github.com/Arasz/ai-badger --skill mcp-tool-surface-testing-arasz

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? When an MCP server's tool surface changes or its documentation drifts from the live contract, there is no systematic way to verify that every exported tool actually works as documented. This Skill provides a disciplined expectations-first audit method so tool testing produces verifiable findings instead of exploratory guessing. ## Core Features & Use Cases - Expectations-first audit: Write an expectations table (tool, expected behaviour, expected response shape) from docs, live schemas, and mcp-index intents before making any call, then compare actual responses against it. - Destructive-tool safety controls: Use a dedicated test project, positive and negative controls for delete/sweep/share tools, and verified zero-residue cleanup. - Dependency-ordered execution with verdicts: Batch independent calls, sequence dependent ones, and record PASS/PARTIAL/FAIL verdicts plus a findings section covering docs drift and response-shape inconsistencies. - Use Case: A server's surface grew from 19 to 21 tools. Run the full surface test, commit a results doc to the repo showing 33 PASS / 2 PARTIAL, and fold newly discovered quirks into the server's pitfalls documentation. ## Quick Start Test every tool exported by the memory MCP server and commit a results doc to the repo's work-docs directory.

Frequently Asked Questions about mcp-tool-surface-testing

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test all tools exported by an MCP server?

Enumerate the full tool surface from the mcp-index or host tool catalog, write an expectations table from docs and live schemas before any call, then execute calls in dependency order and record PASS, PARTIAL, or FAIL verdicts in a results doc.

How to safely test destructive MCP tools like delete or sweep?

Use a dedicated test project id, never real project data. Run positive and negative controls such as a bogus hash returning zero deletions, then verify cleanup with a final stats call proving zero residue.

Why do MCP tool docs differ from actual responses?

Documentation tables lag the live contract: response keys get renamed, schemas gain undocumented parameters, and access-tier claims are deployment-dependent. The live schema and response are the authoritative contract, so flag drift as a finding.

Can I automate MCP tool surface testing as a regression test?

Yes, the references include an xunit E2E parity recipe using McpClient that asserts the exact tools/list set and round-trips uncovered tools. Drop intermittently flaky tools rather than letting them poison CI runs.

What are the limitations of black-box MCP surface testing?

It verifies observable behaviour, not internal correctness, and access-tier results depend on the deployment's mode. Config-gated tools fail with typed errors until configured, which are valid results requiring remedy and retest.