skills.rest
agent0ai
agent0aiOFFICIAL
Explore all skills by this author →
Website𝕏@Agent0aiGitHub

host-computer-use

Controls the connected host computer desktop through the computer_use_remote tool.

19.1k|3.8k|Updated Jun 10, 2024
Search Tags:#computer use#screenshot#remote control#desktop automation#Accessibility Tree#gui interaction#host machine
One-click install
npx skills add https://github.com/agent0ai/agent-zero --skill host-computer-use
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: host-computer-use
Source: https://github.com/agent0ai/agent-zero/tree/main/plugins/_a0_connector/skills/host-computer-use
Command: npx skills add https://github.com/agent0ai/agent-zero --skill host-computer-use

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It lets an agent operate the user's real local computer—taking screenshots, clicking, typing, scrolling, and inspecting native app UI—through a connected A0 CLI or Launcher host gateway, without confusing the host machine with the internal Agent Zero Docker desktop.

Core Features & Use Cases

  • Remote desktop control: Start sessions, capture screenshots, move, click, scroll, send keys, and type text on the host machine via the computer_use_remote tool.
  • Structured UI targeting: Use list_windows, get_window_state, and element_action with background dispatch when the backend advertises accessibility-tree support (AT-SPI on Linux, Accessibility on macOS, UIA on Windows).
  • Clear boundaries: Distinguishes host control from the internal Linux desktop and from browser automation, directing browser tasks to the browser tool instead.
  • Use Case: Ask the agent to open a native desktop application on your Ubuntu Wayland machine, navigate its menus, and fill in a dialog, with each step verified against fresh screenshots.

Quick Start

Ask the agent to take a screenshot of your local computer screen and click a specific button in a native application window.

Frequently Asked Questions about host-computer-use

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I control my local computer desktop from an AI agent?▼

Connect your machine through A0 CLI or the A0 Launcher host gateway, enable computer use with /computer-use on, then the agent uses the computer_use_remote tool to start a session, capture screenshots, click, type, and scroll on your host screen.

What is the difference between host computer use and the Agent Zero Linux desktop?▼

Host computer use controls your real local machine through the connected CLI, while the Linux desktop skill operates the internal Dockerized XFCE environment. Host screen actions must never fall back to the internal desktop or server shell tools.

Does host computer use work on macOS and Windows?▼

Yes, the tool supports Linux Wayland, macOS, and Windows backends. Each platform exposes structural accessibility targeting (AT-SPI, macOS Accessibility, Windows UIA) when the connected backend advertises the matching features.

Should I use computer use or the browser tool for web tasks?▼

Use the browser tool for web navigation, including host browser control, since it is more reliable and token-efficient than screenshot-driven desktop control. Reserve computer use for desktop and native UI tasks the browser tool cannot express.

Why does computer use say rearm required and stop working?▼

The rearm-required status means the desktop-control backend is installed but not armed. Run /computer-use on in the A0 Launcher chat or A0 CLI and approve the platform permission prompt; the agent must not bypass this with shell or screenshot fallbacks.

Related Skills

Explore More Software Engineering Skills→
kortix-aikortix-ai
20.2k

kortix-computer

Controls connected remote machines via the computer connector for filesystem, shell, and desktop operations.

Official
Intermediate
jholhewresjholhewres
14

execution

Execute shell commands and transfer files via SSH and SCP.

Official
Intermediate
heysuhasheysuhas
2

computer-use

Automate desktop GUI actions via accessibility tree mappings on macOS, Windows, and Linux.

Community
Advanced
ToXMonToXMon

a0-setup-cli

Install and connect the A0 CLI on macOS, Linux, or Windows hosts.

Community
Intermediate