ha-mac-control

What problem does it solve?

It solves the difficulty of reliably operating macOS UI elements (apps, windows, dialogs, menus, Dock, Spaces, and clipboard) when accessibility identifiers are volatile and the UI state changes between steps.

Core Features & Use Cases

• Authorized macOS desktop control with state recovery: performs status checks, app/window/menu/dialog inspection, and action execution with fresh observation to avoid stale UI targets.
• Robust targeting rules for UI automation: uses bundleId/windowId/snapshotId + elementId re-resolution and avoids guessing when multiple elements match.
• Visual positioning and OCR when AX is insufficient: locates visible UI via annotated visual observe, OCR, and pixel-to-screen point mapping, then verifies results after actions.
• Safe workflow verification loops: re-snapshots, re-lists, or inspects dialogs/menus after mutations to confirm the expected state.

Quick Start

Ask the AI to control your Mac to open a specific app, navigate to the UI location you describe (e.g., a Dock item, a Space, a menu command, a dialog button, or a visible button), and then perform the action while it verifies the result.