What problem does it solve? Driving a Mac's GUI programmatically usually means hijacking the user's cursor, keyboard focus, and active Space, which interrupts real work. This Skill lets an AI operate macOS apps in the background using accessibility-tree element indices and scoped screenshots, so automation runs without disturbing the user. ## Core Features & Use Cases - Background GUI automation: Capture screens, click, type, scroll, and drag in any app without raising windows, stealing focus, or switching Spaces. - Element-index targeting: SOM capture mode overlays numbered indices on interactable elements from the accessibility tree, making clicks far more reliable than raw pixel coordinates for any tool-capable model. - Built-in safety guardrails: Hard rules block interacting with permission dialogs, password fields, payment UI, and dangerous typed commands, and treat on-screen content as untrusted. - Use Case: Ask the agent to open Safari in another Space, navigate to a site, fill a search field, and report back the results — all while you keep typing in your editor uninterrupted. ## Quick Start Use the computer_use tool to capture Safari with element overlays, then click element 7 and show me a screenshot of the result.
