skills.rest
dawsonblock
dawsonblockCOMMUNITY
Explore all skills by this author →
GitHub

cua-driver

Automate macOS GUI tasks in the background using cua-driver without foreground activation.

Updated Apr 26, 2026
Search Tags:#accessibility#automation-testing#cua-driver#macos-automation#background-drive#window-state
One-click install
npx skills add https://github.com/dawsonblock/HERMY --skill cua-driver
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cua-driver
Source: https://github.com/dawsonblock/HERMY/tree/main/cua-main/libs/cua-driver/Skills/cua-driver
Command: npx skills add https://github.com/dawsonblock/HERMY --skill cua-driver

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

macOS GUI automation runs in the background using cua-driver to drive apps without forcing foreground activation, preserving user context while performing tasks.

Core Features & Use Cases

  • Hidden launch, AX-tree snapshotting, and element_index driven actions that keep the target app in the background.
  • Flexible capture modes (som/ax/vision) with per-pid routing, verification via after-state snapshots, and robust safety checks.
  • Typical workflows include automating Finder, Calculator, or browser apps without visual interruption, enabling testing and automation scenarios.

Quick Start

Launch the target app with launch_app, snapshot with get_window_state, and perform actions via element_index while keeping the frontmost context unchanged.

Frequently Asked Questions about cua-driver

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS GUI tasks without bringing the app to the foreground?▼

macOS GUI automation runs in the background using cua-driver to drive apps without forcing foreground activation, preserving user context while performing tasks. It enforces a no-foreground contract to keep the frontmost context intact.

How do I use an accessibility tree to perform actions on background macOS apps?▼

Accessibility tree snapshotting via get_window_state captures the AX-tree, allowing element_index driven actions to target specific elements while keeping the target app in the background without visual interruption.

Does cua-driver support multiple capture modes for macOS automation testing?▼

cua-driver supports flexible capture modes including som, ax, and vision for macOS automation testing. It routes actions per-pid and verifies results via after-state snapshots with robust safety checks.

What is the best way to automate Finder or Calculator without disrupting my current window?▼

Automating Finder or Calculator without visual interruption is best achieved through hidden launch and per-pid input delivery, keeping the user's frontmost context unchanged while performing structured, auditable actions.

What are the limitations of background macOS automation regarding window state?▼

Background macOS automation requires safety validation across capture modes to coordinate launch-state retrieval and per-pid input delivery, but enforces a strict no-foreground contract that may limit apps requiring active visual focus.

Related Skills

Explore More Software Engineering Skills→
devMoezdevMoez
1

macos-computer-use

Drive macOS desktop interactions via screenshots and AX element indexes.

Community
Intermediate
JordiNodeJSJordiNodeJS

webapp-testing

Automate UI/UX testing of web apps using Chrome DevTools MCP.

Community
Advanced
MarbleSodasMarbleSodas

macos-computer-use

Automate macOS desktop apps via background GUI with element-index targeting.

Community
Intermediate
RheasilviaRheasilvia

computer-use

Automate native desktop application interactions via accessibility trees and input events.

Community
Advanced