skills.rest
Piaoxuemoli
PiaoxuemoliCOMMUNITY
Explore all skills by this author →
GitHub

macos-computer-use

Automate macOS desktop interactions with screenshots, clicks, and typing.

Updated Jun 15, 2026
Search Tags:#macos#interaction#background#screenshot#gui#desktop automation
One-click install
npx skills add https://github.com/Piaoxuemoli/qoobeeHermes --skill macos-computer-use-piaoxuemoli
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: macos-computer-use
Source: https://github.com/Piaoxuemoli/qoobeeHermes/tree/main/hermes/skills/apple/macos-computer-use
Command: npx skills add https://github.com/Piaoxuemoli/qoobeeHermes --skill macos-computer-use-piaoxuemoli

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires cua-driver, and includes scripts (resource) components.

What problem does it solve?

This Skill allows you to interact with a macOS desktop in the background without interrupting the user, enabling automation of tasks like screenshots, clicks, and scrolling.

Core Features & Use Cases

  • Background Interaction: Perform actions like screenshots, clicks, and scrolling on macOS without user interruption.
  • Screenshot Modes: Capture screenshots with or without element overlays and AX trees for different use cases.
  • Element Interaction: Click, double-click, right-click, middle-click, and drag elements by index or coordinate.
  • Scrolling: Scroll within a viewport or to a specific point under an element.
  • Text Input and Keyboard Shortcuts: Type text and use keyboard shortcuts programmatically.
  • Focus Management: List and focus on running applications without raising windows.
  • Use Case: Automate complex workflows like filling out forms in a separate Space or running automated tests on a desktop app.

Quick Start

Take a screenshot of the Safari app and click the 'Sign In' button.

Frequently Asked Questions about macos-computer-use

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS desktop interactions in the background without interrupting the user?▼

To automate macOS desktop interactions in the background, you can use this Skill to perform actions like capturing screenshots, simulating clicks, and typing without raising windows or interrupting the user. It leverages the cua-driver for low-level GUI interaction.

Can I capture a screenshot with element overlays and AX trees on macOS?▼

Yes, you can capture screenshots with or without element overlays and AX trees on macOS. This Skill supports various screenshot modes to facilitate different desktop automation and GUI testing use cases.

Do I need cua-driver to simulate clicks and typing on a macOS desktop?▼

Yes, you need cua-driver to simulate clicks and typing on a macOS desktop. It provides the necessary low-level interaction with the macOS GUI and system accessibility features required for background desktop operations.

What is the best way to automate GUI testing for a desktop app in a separate macOS Space?▼

The best way to automate GUI testing in a separate macOS Space is using this Skill to manage focus, list running applications, and interact with elements without raising windows. It enables complex workflows like form filling and automated testing.

How do I scroll to a specific point or drag elements by index during macOS desktop automation?▼

To scroll to a specific point or drag elements during macOS desktop automation, this Skill provides precise viewport scrolling and element interaction capabilities. You can click, double-click, right-click, and drag elements by index or coordinate programmatically.

What are the limitations of automating text input and keyboard shortcuts on macOS?▼

While automating text input and keyboard shortcuts on macOS is supported, limitations depend on the target application's accessibility features. The cua-driver must interface properly with system accessibility APIs to execute programmatic typing and shortcuts successfully.

Related Skills

Explore More Software Engineering Skills→
519lab519lab
2

macos-computer-use

Automate macOS GUI interactions without user focus.

Official
Intermediate
ZardLi1115ZardLi1115
4

macos-computer-use

Automate macOS desktop GUI tasks via screenshots and accessibility element indices.

Community
Intermediate
zm2231zm2231

workflow-orchestrator

Orchestrate multi-step workflows with dependencies and conditional logic.

Community
Advanced
mikeyobrienmikeyobrien
369

notification

Display native macOS notifications with title, message, and optional sound.

Community
Basic