What problem does it solve?
Enable safe, programmatic access to a user's local machine so an AI or automation agent can read/write files, run shell commands, capture screenshots, and interact with GUI elements without manual step-by-step user intervention.
Core Features & Use Cases
- File I/O: Read, write, and list files and directories on the user's computer.
- Remote Command Execution: Run shell commands with controlled timeouts and receive structured stdout/stderr results.
- Desktop Automation: Capture screenshots and cursor regions, move/click/drag the mouse, type text, and press key combinations.
- Accessibility-driven UI Automation: Inspect the accessibility tree, search UI elements, focus elements, set values, and trigger AX actions for reliable cross-resolution interactions.
- Operational Safety: Uses an API-backed tunnel with permission requests, JSON outputs for deterministic parsing, and explicit error messages for offline or permission states.
- Use Case Example: Troubleshoot a user issue by listing open windows, taking a screenshot, locating a button via the accessibility tree, and triggering a click or AXPress to reproduce and fix the problem.
Quick Start
Take a screenshot of my desktop, list open windows, and read the file at ~/Documents/report.txt.