skills.rest
melandlabs
melandlabsOFFICIAL
Explore all skills by this author →
𝕏@AlloomiAIGitHub

cua-driver

Automate macOS application interactions via accessibility APIs and event simulation.

909|43|Updated Apr 10, 2026
Search Tags:#automation#macos#screenshot#app interaction#element-index
One-click install
npx skills add https://github.com/melandlabs/openloomi --skill cua-driver-melandlabs
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cua-driver
Source: https://github.com/melandlabs/openloomi/tree/main/skills/cua-driver
Command: npx skills add https://github.com/melandlabs/openloomi --skill cua-driver-melandlabs

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires cua-driver, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill enables automatic interaction with macOS applications, allowing the AI to operate and automate tasks within these apps without user intervention.

Core Features & Use Cases

  • Element-Indexed Actions: Perform precise actions like clicks, typing, and scrolling within applications by referencing elements by their index in the accessibility tree.
  • Snapshots Before & After: Take screenshots before and after actions to verify the outcome, ensuring actions are performed correctly.
  • Web Application Support: Handle interactions with web applications embedded in macOS apps, such as Chrome and Safari, through alternative methods due to limited accessibility support.
  • Use Case: Automate the process of opening a file in an application like TextEdit, clicking a button to save, and then closing the application without any user interaction.

Quick Start

Use the cua-driver skill to open the 'TextEdit.app', click on the 'File' menu, select 'Save As', type 'output.txt' into the file name field, and click 'Save'.

Frequently Asked Questions about cua-driver

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS app interactions without disrupting my active workflow?▼

You can automate macOS app interactions in the background by using native accessibility APIs and event simulation to perform precise actions without disrupting your active workflow. This allows the AI to operate apps seamlessly while you continue working.

What are element-indexed actions for macOS automation?▼

Element-indexed actions allow you to perform precise clicks, typing, and scrolling within macOS applications by referencing elements through their specific index in the accessibility tree. This ensures targeted control over UI components during automation.

Do I need special system permissions to automate macOS applications?▼

Yes, automating macOS applications requires granting accessibility and screen recording permissions to the cua-driver CLI. These system permissions are necessary to utilize native macOS accessibility APIs and capture screenshots for action verification.

Can I automate web applications embedded in macOS apps like Chrome or Safari?▼

Yes, you can automate web applications embedded in macOS apps like Chrome and Safari using alternative interaction methods. This approach is necessary because these web views have limited native accessibility support compared to standard macOS applications.

How do I verify if an automated action in a macOS app was performed correctly?▼

You can verify automated macOS app actions by taking screenshots before and after the execution. These snapshots allow you to visually confirm the outcome and ensure that clicks, typing, or scrolling actions were performed correctly.

What are the limitations of using accessibility APIs for macOS web automation?▼

The primary limitation is that web content within macOS apps has restricted accessibility API support, requiring alternative methods for automation. Standard native macOS apps offer better accessibility tree integration compared to embedded web views.

Related Skills

Explore More Software Engineering Skills→
RuifengFuRuifengFu

cua-driver

Automates macOS app interactions via the Accessibility tree using cua-driver.

Community
Advanced
hungthinh04hungthinh04

macos-computer-use

Control macOS desktop apps in the background without stealing focus.

Community
Advanced
Ayushman-11Ayushman-11

ui-visual-validator

Validate UI modifications through visual regression testing and design system compliance.

Community
Advanced
dvdsgldvdsgl
54

imessage

Read and send iMessages via sqlite3 and AppleScript on macOS.

Community
Advanced