skills.rest
shalevamin
shalevaminCOMMUNITY
Explore all skills by this author →
GitHub

tua-computer-use

Controls a paired Mac with screenshots, mouse, and keyboard actions for desktop automation.

Updated Mar 7, 2026
Search Tags:#accessibility#macos automation#computer use#screenshot#desktop automation#keyboard input#mouse control
One-click install
npx skills add https://github.com/shalevamin/Tau_agent --skill tua-computer-use-shalevamin
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tua-computer-use
Source: https://github.com/shalevamin/Tau_agent/tree/main/tau-agent-main/skills/tua-computer-use
Command: npx skills add https://github.com/shalevamin/Tau_agent --skill tua-computer-use-shalevamin

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Automating tasks that require the real visible macOS UI—desktop apps, logged-in browser sessions, or interfaces that pure browser automation cannot reach—requires direct control of the screen, mouse, and keyboard, which this Skill provides through a paired node. ## Core Features & Use Cases - Screenshot-Based Operation Loop: Captures the screen, decides the next action, executes it, and verifies the result with a fresh screenshot after every step. - Mouse and Keyboard Control: Supports move, click, double-click, scroll, right-click, free text typing, and shortcuts like command+c, command+v, and command+l. - Coordinate Rescaling: Translates screenshot image coordinates into real screen coordinates using imageWidth and imageHeight parameters. - Use Case: Ask the agent to open a desktop application, navigate its menus, fill in a dialog, and confirm the result visually—useful when no API or browser automation path exists. ## Quick Start Ask the agent to take a screenshot of your Mac and then click a specific button or type text into the currently focused application.

Frequently Asked Questions about tua-computer-use

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate mouse clicks and keyboard input on macOS with an AI agent?▼

Use the nodes tool actions computer_screenshot, computer_mouse, computer_type, and computer_keypress. Start with a screenshot, take one small action at a time, and capture a new screenshot after each action to verify the result.

When should I use computer control instead of browser automation?▼

Prefer browser-use for pure browser automation first. Use computer control when the task needs the real visible UI, native desktop apps, or a logged-in browser session that browser-only automation cannot complete reliably.

Why do screenshots fail on macOS computer automation?▼

Screenshot failures mean the Screen Recording permission is missing for the paired node. Grant Screen Recording permission in macOS System Settings, then retry the computer_screenshot action.

Why are mouse and keyboard actions not working on my Mac?▼

Mouse and keyboard failures indicate the Accessibility permission is missing. Enable Accessibility access for the node in macOS System Settings under Privacy and Security, then retry the action.

How do screenshot coordinates map to real screen positions?▼

computer_screenshot returns image dimensions plus coordinateWidth and coordinateHeight for the click space. Pass imageWidth and imageHeight back into computer_mouse, and the node rescales them to real screen coordinates automatically.

Related Skills

Explore More Software Engineering Skills→
s1366560s1366560
5

peekaboo

Automate macOS UI interactions via command-line with screenshots, input, and window management.

Community
Advanced
Cortexa-LLCCortexa-LLC

orchestrator

Delegate independent subtasks to specialized agents for parallel execution.

Official
Advanced
leogurjaleogurja
1

computer-use

Inspect and control visible local app windows through the Orca CLI.

Community
Intermediate
ammonfifeammonfife

bigmac-e2b

Create and manage BIGMAC E2B sandboxes for code execution and browser automation.

Community
Advanced