computer-use

Name: computer-use
Availability: InStock
Author: cerisonautomation

Control macOS desktop UI via accessibility tree and element-targeted actions.

Updated Aug 23, 2026

Search Tags:#macos #agentic workflows #desktop automation #gui control #accessibility tree #proxy tools #screen observation

Ready to run

npx skills add https://github.com/CerisonAutomation/clud --skill computer-use-cerisonautomation

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: computer-use
Source: https://github.com/CerisonAutomation/clud/tree/main/assistant/src/config/bundled-skills/computer-use
Command: npx skills add https://github.com/CerisonAutomation/clud --skill computer-use-cerisonautomation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill eliminates slow, error-prone manual UI interactions by letting the assistant observe the screen and perform precise desktop actions on macOS through accessibility-driven controls.

Core Features & Use Cases

Visual screen observation: Capture the current UI state (accessibility tree with stable element IDs) and optionally a screenshot before acting.
Direct GUI control: Click, double-click, right-click, type text, press keys/shortcuts, scroll, drag-and-drop, and open or switch apps.
Safe session flow: Use explicit completion and response tools to end the computer-use session cleanly after the user goal is met.

Quick Start

Ask the assistant to control your macOS desktop to complete your task by observing the screen, identifying the right UI elements by ID, and then performing the required click/type/keyboard steps.

Frequently Asked Questions about computer-use

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate macOS desktop workflows for apps without an API?▼

Yes, you can automate GUI form-filling tasks on macOS by using this Skill to observe the current screen state, identify input fields through the accessibility tree, and execute targeted typing and clicking actions. It handles visual workflows by interacting directly with on-screen elements.

How does AI observe the screen to perform desktop automation?▼

To start automating UI interactions with this Skill, ask the assistant to observe your macOS screen, identify the correct UI elements by their accessibility IDs, and perform the required click, type, or keyboard shortcut steps. It provides proxy-executed tools for direct GUI control and requires a host execution target.

Do I need a host execution target to control macOS desktop actions?▼

Yes, this Skill supports running AppleScript to handle complex macOS desktop automation tasks alongside observing the accessibility tree and executing UI actions. You can use the run AppleScript tool to perform system-level operations that complement standard GUI control workflows.

How do I end a macOS desktop automation session cleanly?▼

To end a macOS desktop automation session cleanly, use the explicit completion and response tools provided by this Skill to trigger a deterministic done/response flow. This ensures the computer-use session terminates safely after your user goal is met.