skills.rest
islee23520
islee23520COMMUNITY
Explore all skills by this author →
Website𝕏@iz_23520GitHub

cua-driver

Drive native macOS apps via accessibility tree, mouse, keyboard, and screenshots.

5|Updated May 17, 2026
Search Tags:#accessibility#macos#computer use#desktop automation#gui control#screenshot verification
One-click install
npx skills add https://github.com/islee23520/lfg --skill cua-driver-islee23520
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cua-driver
Source: https://github.com/islee23520/lfg/tree/main/skills/cua-driver
Command: npx skills add https://github.com/islee23520/lfg --skill cua-driver-islee23520

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill removes the friction of manually operating native macOS applications by letting an agent drive the interface directly, even when the target app is not frontmost.

Core Features & Use Cases

  • Native App Automation: Launch apps, inspect the accessibility tree, click elements, type text, scroll, and press keys.
  • Observe-Act-Verify Workflow: Capture a snapshot before acting, perform the interaction, then re-snapshot to confirm the result.
  • Backgrounded Control: Work on hidden, minimized, or off-Space windows so the user's foreground editor or browser stays undisturbed.
  • Real-World Use Cases: Open files in TextEdit, navigate Finder, confirm a save dialog in Numbers, or operate creative and productivity tools on macOS.

Quick Start

Ask the agent to use cua-driver to open the requested macOS app, inspect its window state, perform the needed interaction, and verify the result with a follow-up snapshot.

Frequently Asked Questions about cua-driver

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate native macOS applications in the background without bringing them to the front?▼

macOS desktop automation is achieved by operating the accessibility tree, mouse, keyboard, and screenshots through a dedicated driver. This allows an agent to launch apps, select elements, and verify results even when the target window is backgrounded.

What is the observe-act-verify workflow for GUI control on macOS?▼

The observe-act-verify workflow for GUI control captures a screenshot before acting, performs the interaction, then re-snapshots to confirm the result. This loop ensures desktop automation tasks are verified safely.

Do I need accessibility and screen recording permissions for macOS desktop automation?▼

Yes, macOS desktop automation requires Accessibility and Screen Recording permissions, plus a running daemon. These grants allow the agent to inspect the accessibility tree and capture screenshots safely.

Can I use desktop automation to interact with minimized or off-Space windows on macOS?▼

Yes, desktop automation can interact with minimized or off-Space windows on macOS. By operating the accessibility tree directly, the agent controls backgrounded apps without disturbing your active Space.

What are the limitations of using the accessibility tree for macOS desktop automation?▼

Limitations of accessibility tree automation include requiring explicit Accessibility and Screen Recording permissions and a running daemon. It may also fail if target apps lack proper accessibility labels or do not expose UI elements correctly.

Related Skills

Explore More Software Engineering Skills→
zaiyemeiyou404zaiyemeiyou404

macos-computer-use

Automate macOS GUI actions via cua-driver without disrupting the desktop.

Community
Advanced
Wike-CHIWike-CHI
2

desktop-control

Automate mouse, keyboard, and window operations with OpenCV-based image matching.

Community
Advanced
wangqianCAIwangqianCAI

peekaboo

Automate macOS UI interactions via accessibility and screen recording APIs.

Community
Advanced
dvdsgldvdsgl
54

imessage

Read and send iMessages via sqlite3 and AppleScript on macOS.

Community
Advanced