skills.rest
siddsachar
siddsacharCOMMUNITY
Explore all skills by this author →
GitHub

computer_use_guide

Guides native desktop automation through scoped captures, token-bound actions, and retry rules.

1.5k|172|Updated Mar 6, 2026
Search Tags:#computer use#desktop automation#native apps#retry policy#UI interaction#delivery mode#target capture
One-click install
npx skills add https://github.com/siddsachar/row-bot --skill computer-use-guide
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: computer_use_guide
Source: https://github.com/siddsachar/row-bot/tree/main/tool_guides/computer_use_guide
Command: npx skills add https://github.com/siddsachar/row-bot --skill computer-use-guide

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Native desktop automation fails when agents replay unsafe actions, switch tools mid-task, or act on stale targets. This Skill defines strict operating rules for the computer_use tool so clicks, typing, and captures stay deterministic and recoverable.

Core Features & Use Cases

  • Scoped Targeting: Enforces one app-scoped capture with an exact list_apps name, eliminating parallel discovery and ambiguous window selection.
  • Token-Bound Actions: Governs target_id and token lifecycles per generation, with explicit rules for stale refusals, target_gone, and lease expiry.
  • Delivery Mode Escalation: Starts with delivery_mode=auto and permits foreground delivery only after fresh evidence and structured recommendations.
  • Use Case: When automating a native desktop app, follow these rules to click a button once, verify the driver_verdict, and escalate to Take over instead of blindly retrying after repeated stale refusals.

Quick Start

Use the computer use guide to click the Save button in the currently running desktop application and verify the result.

Frequently Asked Questions about computer_use_guide

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate clicks in a native desktop application?▼

Start with one app-scoped capture using an exact name from list_apps, then dispatch a single click against the returned target_id. Issue one Computer Use call at a time and read the driver_verdict before deciding the next step.

When should I use Computer Use instead of browser automation?▼

Prefer Browser for managed pages and Computer Use for native targets, and never switch silently between them. Computer Use is the correct choice when the target is a native application window rather than a web page.

What should I do after a stale target refusal?▼

Capture the exact same target once and retry the same exact action once. If the stale refusal repeats, stop or request Take over instead of switching action families or delivery engines.

Can I retry a typing action that timed out?▼

No. Text insertion, accepted or uncertain mutations, timeouts, disconnects, and ended sessions must never be replayed, since the native state may already have changed.

When is foreground delivery mode allowed?▼

Use delivery_mode=foreground only for click, double-click, right-click, type, key, or scroll after fresh exact-target evidence and a prior structured recommendation or refusal. A second refusal requires Take over.

Related Skills

Explore More Software Engineering Skills→
ZhangHanDongZhangHanDong
37

makepad-screenshot

Capture and analyze Makepad app UI screenshots for visual debugging.

Community
Intermediate
Priivacy-aiPriivacy-ai
1.6k

spk-run-verdict-capture

Records work-package review verdicts through the deterministic event-log seam in Spec Kitty.

Official
Intermediate
pmcdowallpmcdowall

macos-computer-use

Control macOS desktop interfaces without stealing user input focus.

Community
Advanced
ShopifyShopify
7.2k

agent-device

Control iOS simulators and Android devices via a unified command-line interface.

Official
Advanced