skills.rest
alex8224
alex8224COMMUNITY
Explore all skills by this author →
GitHub

cua-driver

Automates Windows desktop applications via accessibility trees, screenshots, and background input delivery.

7|2|Updated Jul 21, 2026
Search Tags:#windows automation#desktop automation#mcp tools#gui testing#accessibility tree#uia#background input
One-click install
npx skills add https://github.com/alex8224/synapse-agent --skill cua-driver-alex8224
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cua-driver
Source: https://github.com/alex8224/synapse-agent/tree/main/skills/cua-driver
Command: npx skills add https://github.com/alex8224/synapse-agent --skill cua-driver-alex8224

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Operating real Windows desktop applications programmatically is hard: windows may lack automation APIs, background input gets dropped, and clicks often fail silently. This Skill drives the local Windows desktop through the cua-driver MCP tools, letting you enumerate windows, read accessibility trees and screenshots, click, type, scroll, and verify that actions actually took effect. ## Core Features & Use Cases - Snapshot-bound element addressing: Capture a window's accessibility tree and screenshot with cua_get_window_state, then act on elements by index tied to that snapshot, avoiding stale-reference errors. - Background-first input delivery: Click, type, and set control values without stealing focus or moving the real mouse, with a structured escalation ladder to foreground mode only when required (e.g., Chromium keystrokes). - Action verification and pixel fallback: Re-read window state to confirm effects, and fall back to screenshot-based pixel coordinates for self-drawn UIs like WeChat that expose no accessibility tree. - Use Case: Ask the agent to open Calculator via its AUMID, read the window state, click buttons 4, +, 2, =, then re-read the result field to confirm it displays 6 — all without touching your real mouse. ## Quick Start Use the cua-driver skill to open Windows Calculator, compute 40 + 2, and verify the result shown on screen.

Frequently Asked Questions about cua-driver

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate a Windows desktop application with an AI agent?▼

List windows with cua_list_windows, capture the accessibility tree and screenshot via cua_get_window_state, then act on elements by index using cua_click or cua_set_value. Always re-read the window state afterward to verify the action took effect.

How do I click a UI element using an accessibility tree?▼

Call cua_get_window_state to get a snapshot_id and element indices, then pass both element_index and snapshot_id to cua_click. Element indices are snapshot-bound, so reusing an old snapshot returns an element_token is stale error.

Why does background keyboard input fail in Chrome or Edge?▼

Chromium's Chrome_WidgetWin_1 window class drops background keyboard and text synthesis events. Use cua_set_value for form filling in the background, and retry only the submit keystroke with delivery_mode foreground after a background_unavailable error.

What can I automate when an app has no accessibility tree?▼

Self-drawn UIs like WeChat report element_count 0, so switch to the pixel path: save a screenshot with screenshot_out_file, zoom to read coordinates, then use cua_click with window-local x,y and cua_type_text for input.

Does cua-driver require admin rights or move the real mouse?▼

No administrator rights are needed, and the default background delivery mode never raises windows or moves the physical mouse. It requires an interactive Windows session and the cua server enabled in the Synapse MCP configuration.

Why did my click report success but nothing changed on screen?▼

UIA Invoke returns effect unverifiable, which means neither success nor failure. Take a fresh cua_get_window_state snapshot and read the application state, and check that you did not mix screen-absolute frame coordinates with window-local click coordinates.

Related Skills

Explore More Software Engineering Skills→
theflysurfertheflysurfer

julien-mcp-excel-win32com

Install the Excel Win32COM MCP server into .mcp.json configuration.

Community
Advanced
githubgithub
38.5k

automate-this

Analyzes screen recordings to reconstruct workflows and generate working automation scripts.

Official
Advanced
pkuppenspkuppens
1

run-validation

Execute validation steps from a Markdown file and compare outcomes.

Community
Advanced
melandlabsmelandlabs
909

agent-browser

Automate web interactions like form submissions and data extraction via CLI.

Official
Advanced