find-widget

Scan visible game UI widgets and return compact CLICK_WIDGET results.

Updated Aug 23, 2026
One-click install
npx skills add https://github.com/Tsangares/manny_mcp --skill find-widget
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: find-widget
Source: https://github.com/Tsangares/manny_mcp/tree/main/.claude/skills/find-widget
Command: npx skills add https://github.com/Tsangares/manny_mcp --skill find-widget

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps locate a clickable widget in the game UI by text or name using Haiku, without flooding the main context with every widget.

Core Features & Use Cases

  • Scan all visible widgets with scan_widgets() to gather IDs, text, names, bounds, and actions.
  • Filter results by matching the user’s query against the text or name fields and by actions (e.g., Cook, Bank, Talk-to).
  • Return compact results including widget_id, text/name, actions, and bounds, plus guidance for initiating a CLICK_WIDGET command.
  • Use case: locate a specific widget like "Raw shrimps" with the Cook action and obtain its widget_id and bounds to click.

Quick Start

Use the find-widget skill to locate a UI element by text or name and then click it using the widget_id. Start by scanning widgets with scan_widgets(), apply a text/name search, optionally filter by actions, and finally issue CLICK_WIDGET with the found widget_id.

Frequently Asked Questions about find-widget

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find a clickable UI widget by text in game automation?

To find a clickable UI widget by text, scan all visible widgets using scan_widgets(), filter the results by matching text or name fields, and retrieve the widget_id and bounds needed to execute a click command. This approach minimizes context flooding by returning only essential fields.

How does widget scanning work with Haiku to locate game UI elements?

Widget scanning with Haiku works by gathering IDs, text, names, bounds, and actions from all visible widgets, then filtering them against your search query and desired actions to return compact results ready for clicking. Haiku processes the scan efficiently to avoid flooding the main context.

Do I need Haiku as my execution model to use find-widget?

Yes, you need Haiku as your execution model to use find-widget. The skill specifically requires Haiku to scan widgets and filter results efficiently, ensuring that only compact data like widget_id, text, actions, and bounds are returned for clicking.

What's the best way to click a specific game UI element like Raw shrimps with a Cook action?

The best way to click a specific game UI element like Raw shrimps is to scan widgets, filter by the text "Raw shrimps" and the "Cook" action, then use the returned widget_id and bounds to issue a CLICK_WIDGET command directly.

Can I filter visible widgets by actions like Cook, Bank, or Talk-to?

Yes, you can filter visible widgets by actions like Cook, Bank, or Talk-to. After scanning widgets with scan_widgets(), you apply a text or name search and optionally filter by specific actions to narrow down the results to the correct clickable widget.

What data does the widget search return to prevent context flooding?

The widget search returns only essential compact fields to prevent context flooding: widget_id, text or name, actions, and bounds. These fields provide everything needed to construct a ready-to-use CLICK_WIDGET command for the selected widget.