gateway-desktop

Name: gateway-desktop
Availability: InStock
Author: bairui

Automate macOS desktop GUI tasks with screenshot-guided actions.

12|3|Updated Mar 6, 2026

Search Tags:#automation #accessibility #macos #screenshot #gui-automation #gateway-desktop

Ready to run

npx skills add https://github.com/baibairui/WoClaw --skill gateway-desktop

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: gateway-desktop
Source: https://github.com/baibairui/WoClaw/tree/main/.codex/skills/gateway-desktop
Command: npx skills add https://github.com/baibairui/WoClaw --skill gateway-desktop

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires @nut-tree-fork/nut-js, and includes scripts (resource) components.

What problem does it solve?

When tasks require operating desktop applications on macOS and need screenshot-guided GUI actions.

Core Features & Use Cases

Start, activate, and control macOS apps using scripted GUI actions with screenshot validation.
Support an observe -> act -> observe loop to validate outcomes and drive interactive workflows.
Handles local macOS execution, required accessibility and screen recording permissions, and stores visual artifacts under .codex/artifacts/desktop.

Quick Start

Start by running node ./scripts/gateway-desktop.mjs observe to capture the current desktop state.

Frequently Asked Questions about gateway-desktop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate macOS desktop GUI tasks using screenshot validation?▼

Automating macOS desktop GUI tasks with screenshot validation uses an observe-act-observe loop. The Skill captures the current desktop state, executes scripted clicks and typing, and validates the visual outcome to drive interactive workflows.

What macOS permissions are required to run scripted GUI automation?▼

Scripted GUI automation on macOS requires local execution with both accessibility and screen recording permissions. These permissions allow the automation scripts to capture screen states and send control inputs to native applications.

Can I launch and control native macOS apps through visual scripting?▼

Yes, you can launch and control native macOS apps through visual scripting. The automation workflow supports starting, activating, and navigating native applications by applying scripted GUI actions with continuous screenshot validation.

How do I capture the current desktop state before performing GUI actions?▼

To capture the current desktop state before performing GUI actions, run the observe command to initialize the script. This captures a screenshot of the current environment and stores the visual artifacts under the designated desktop directory.

Does this GUI automation approach work with nut-js for native app interactions?▼

Yes, this screenshot-guided GUI automation approach works with nut-js to handle native app interactions. It leverages the library to execute observe-act-observe loops, enabling precise clicking, typing, and screen navigation in real macOS environments.