computer-use

Controls the desktop with mouse, keyboard, and screenshots for UI without APIs.

Updated May 5, 2026
One-click install
npx skills add https://github.com/josippapez/ai-setup --skill computer-use-josippapez
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: computer-use
Source: https://github.com/josippapez/ai-setup/tree/main/claude/plugins/dev-core/skills/computer-use
Command: npx skills add https://github.com/josippapez/ai-setup --skill computer-use-josippapez

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Many tasks involve native app windows, OS dialogs, or browser chrome that expose no API and no DOM, leaving automation tools blind. This Skill drives the real desktop directly with mouse, keyboard, and screenshots so those interfaces become operable. ## Core Features & Use Cases - Desktop control via one MCP tool: Perform screenshots, cursor moves, clicks, drags, scrolling, key presses, and typing through a single deferred tool. - OS and browser chrome automation: Handle file pickers, permission prompts, System Settings, the menu bar, address bar, extension popups, and autofill dropdowns. - Visual verification: Capture real screenshots to confirm what rendered, what has focus, or which window is in front. - Use Case: After a Chrome restart, chrome-devtools calls hang behind the native "Allow remote debugging?" dialog; take a screenshot, click Allow at its coordinates, confirm it is gone, and re-run the blocked call. ## Quick Start Ask the assistant to take a screenshot of the screen and click the Allow button on the remote debugging dialog blocking the browser session.

Frequently Asked Questions about computer-use

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate native app UI that has no API?

Use the computer tool's screenshot, click, key, and type actions to drive the native interface directly. Load the deferred tool first with ToolSearch, then prefer keyboard shortcuts over coordinate clicks since they land more reliably.

How to fix chrome-devtools hanging after a Chrome restart?

Chrome 144+ shows a native "Allow remote debugging?" dialog that blocks every chrome-devtools call. Take a screenshot, left_click Allow at its coordinates, screenshot again to confirm dismissal, then re-run the chrome-devtools call.

When should I use computer-use instead of chrome-devtools?

Use chrome-devtools for anything inside a web page since it reads the DOM and clicks by uid. Use computer-use for browser chrome outside the page, native apps, OS dialogs, and seeing what actually rendered on screen.

Why did my click miss the target on screen?

Screenshots draw a red crosshair at the cursor; compare it to the target and adjust proportionally in large steps. On macOS a click on an unfocused window may only raise it, so screenshot and click again, aiming at element centres.

What are the safety limits of desktop automation?

The tool gives full control of the machine, so drive only the specific step needed and stop. Never click through consent, payment, or destructive dialogs without asking the user first.