find-improvable-errors

Analyze MCP tool error messages in magebench game exports to identify least helpful ones.

64|8|Updated Oct 24, 2025

Search Tags:#debugging #diagnostics #bridge #error-messages #magebench #mcp-errors

Ready to run

npx skills add https://github.com/GregorStocks/mage-bench --skill find-improvable-errors

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: find-improvable-errors
Source: https://github.com/GregorStocks/mage-bench/tree/main/.claude/skills/find-improvable-errors
Command: npx skills add https://github.com/GregorStocks/mage-bench --skill find-improvable-errors

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

The Find Improvable Error Messages Skill identifies MCP tool error messages across game exports that hinder model recovery and proposes targeted improvements to reduce stuck retries.

Core Features & Use Cases

Surface the least helpful error messages by analyzing game exports and computing metrics such as stuck rate, recovery rate, and message volume.
Diagnose why poor messages fail by examining error text, action_type distributions, model performance, and representative failure examples from exports.
Propose concrete fixes by documenting and editing the error-handling strings in the codebase (e.g., BridgeCallbackHandler.java), regenerating MCP tooling, and validating improvements with follow-up analysis.
Provide an end-to-end workflow from data analysis to patch validation in magebench.

Quick Start

Run the analysis against all exported games to surface the least helpful error messages and begin diagnosing improvements.

Frequently Asked Questions about find-improvable-errors

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I find unhelpful MCP error messages in game exports?▼

Analyze game exports to compute stuck rate, recovery rate, and message volume metrics, surfacing the least helpful MCP error messages hindering model recovery.

Why does my model get stuck retrying after MCP tool errors?▼

Poor MCP error messages lack actionable recovery context, causing stuck retries. Diagnose failures by examining error text, action_type distributions, and representative failure examples from game exports.

How do I fix bad MCP error messages in BridgeCallbackHandler.java?▼

Edit error-handling strings in BridgeCallbackHandler.java, regenerate MCP tooling, and validate improvements through the magebench analysis pipeline to fix unhelpful error messages.

Can I run MCP error message analysis across all exported magebench games?▼

Run the analysis against all magebench game exports to surface the least helpful error messages and begin diagnosing improvements to reduce stuck retries.

What is the best way to improve MCP tool error messages for model recovery?▼

An end-to-end workflow: analyze error metrics from exports, diagnose poor message failures, edit codebase strings, regenerate MCP tools, and validate via follow-up analysis.