skills.rest
vellum-ai
vellum-aiOFFICIAL
Explore all skills by this author →
Website𝕏@vellum_aiGitHub

computer-use

Control a macOS desktop via proxy tools to automate UI tasks.

1.0k|158|Updated Feb 7, 2026
Search Tags:#automation#macos#scripting#ui-interaction#desktop-control#proxy-tools
One-click install
npx skills add https://github.com/vellum-ai/vellum-assistant --skill computer-use-vellum-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: computer-use
Source: https://github.com/vellum-ai/vellum-assistant/tree/main/assistant/src/config/bundled-skills/computer-use
Command: npx skills add https://github.com/vellum-ai/vellum-assistant --skill computer-use-vellum-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Controls the macOS desktop through proxy-based action tools to automate UI tasks.

Core Features & Use Cases

  • Proxy execution: Tools are forwarded to the connected macOS client rather than executed locally.
  • Broad action set: Observe screen state, click, type, scroll, drag, open apps, and run AppleScript via the proxy.
  • Onboarding readiness: Preactivated for conversations with a connected desktop client.

Quick Start

Describe the macOS task you want to accomplish, and the skill will execute it on the connected desktop client via proxy tools.

Frequently Asked Questions about computer-use

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS desktop UI interactions like clicking and typing?▼

You can automate macOS desktop UI interactions by using a proxy-driven computer-use skill that forwards click, type, scroll, and drag commands to a connected macOS client for execution. This approach lets you script desktop actions without local execution.

Can I run AppleScript on a remote macOS client to control desktop applications?▼

Yes, you can run AppleScript on a remote macOS client through proxy-based action tools. The skill forwards your AppleScript commands to the connected host client, enabling remote application control and desktop automation.

What is proxy-based desktop control for macOS automation?▼

Proxy-based desktop control is an automation mechanism that forwards tool executions to a connected macOS host rather than running them locally. It allows you to observe screen state, click, type, scroll, and drag remotely.

Do I need a connected desktop client to automate macOS UI tasks?▼

Yes, you need a connected macOS desktop client to execute proxy-based UI automation tasks. The skill is preactivated for conversations with a desktop client and forwards all tool executions to that host for processing.

How do I observe screen state on a macOS desktop through a proxy?▼

To observe macOS screen state, the proxy-based computer-use skill forwards observation requests to the connected desktop client, which captures and returns the current visual state of the desktop for your automation workflow.

Can I open apps and drag elements on macOS through proxy tools?▼

Yes, proxy tools support opening applications and dragging elements on macOS by forwarding these specific actions to the connected desktop client, which executes them directly on the host operating system.

Related Skills

Explore More Software Engineering Skills→
wasay1200wasay1200

freight-imessage-setup

Configure and validate iMessage relays for bidirectional SMS with broker AI.

Community
Advanced
frankhli843frankhli843

peekaboo

Automate macOS UI interactions by capturing, inspecting, and controlling on-screen elements.

Community
Advanced
dvdsgldvdsgl
54

imessage

Read and send iMessages via sqlite3 and AppleScript on macOS.

Community
Advanced
whichguywhichguy

macos-computer-use

Automate macOS desktop interactions with screenshots, mouse, keyboard, and scroll actions.

Community
Advanced