What problem does it solve? Automating the macOS desktop usually means hijacking the user's cursor, keyboard focus, and active Space, which interrupts their work. This Skill drives native Mac apps in the background using accessibility-tree element indices and scoped screenshots, so GUI automation runs without disturbing the user. ## Core Features & Use Cases - Background GUI Automation: Capture screenshots with numbered element overlays (SOM mode), then click, type, scroll, and drag by element index without raising windows or switching Spaces. - Model-Agnostic Operation: Works with any tool-capable model (Codex, GPT, Gemini, or local OpenAI-compatible endpoints) through a single computer_use tool with capture, click, type, key, scroll, and drag actions. - Safety Guardrails: Hard rules against clicking permission dialogs, typing secrets, or following instructions found in screenshots, plus tool-level blocking of dangerous patterns. - Use Case: Ask the agent to fill out a form in a native Mac app like Messages or Finder while you keep typing in your editor; it captures the app's AX tree, clicks elements by index, and verifies each step with follow-up screenshots. ## Quick Start Use the computer_use tool to capture Safari with SOM overlays, then click the sign-in element by its index and show me the result.
