skills.rest
alexaundre
alexaundreCOMMUNITY
Explore all skills by this author →
GitHub

desktop

Automate macOS UI interactions via local OCR and input emulation.

Updated Aug 27, 2026
Search Tags:#ocr#macos#wechat#osascript#desktop-automation#screen-reading#cliclick
One-click install
npx skills add https://github.com/alexaundre/mycc --skill desktop-alexaundre
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: desktop
Source: https://github.com/alexaundre/mycc/tree/main/.claude/skills/desktop
Command: npx skills add https://github.com/alexaundre/mycc --skill desktop-alexaundre

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires cliclick, pyobjc.

What problem does it solve?

Removes the need to upload full screenshots or manually operate the macOS UI by providing local, low‑token OCR-based perception and scripted input control so the assistant can find text on screen, move the cursor, click buttons, and type reliably.

Core Features & Use Cases

  • Local cursor-proximate OCR using the macOS Vision framework to detect visible text and return clickable coordinates.
  • Input emulation via cliclick and osascript for mouse movement, clicks, key presses, and keyboard shortcuts with verification.
  • Messaging automation and monitoring workflows for apps like WeChat that include read/send scripts and atomic trigger detection.
  • Safety rules and fallbacks: staged permission levels, prohibition on passwords/payments/Terminal control, and screenshot + visual-analysis fallback when OCR fails.
  • Use case: ask the assistant to locate and click a "保存" button, fill a text field, or send a WeChat message and verify delivery via OCR.

Quick Start

Ask CC to use the desktop skill to locate and click the "保存" button in the current macOS window.

Frequently Asked Questions about desktop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS UI interactions without uploading full screenshots?▼

Automate macOS UI interactions using local OCR and input emulation. The skill uses the macOS Vision framework to detect visible text and return clickable coordinates, then uses cliclick and osascript for mouse movement, clicks, and typing without uploading full screenshots.

Can I use local OCR to locate and click a specific button in a macOS app?▼

Yes, local OCR can locate and click specific buttons in macOS apps. The skill uses cursor-proximate OCR via the macOS Vision framework to detect on-screen text and return precise coordinates, allowing cliclick to move the cursor and click targets like a "保存" button.

Does this macOS automation approach work with WeChat for sending and monitoring messages?▼

Yes, this macOS automation approach works with WeChat for sending and monitoring messages. It includes messaging automation workflows with read and send scripts, atomic trigger detection, and verification of message delivery via local OCR.

What are the safety limitations when using osascript and cliclick for desktop automation?▼

Safety limitations for desktop automation include staged permission levels and strict prohibitions. The skill forbids manipulating passwords, processing payments, and controlling Terminal, while providing a screenshot and visual-analysis fallback when local OCR fails.

Do I need cliclick and pyobjc installed to automate desktop tasks on macOS?▼

Yes, you need cliclick and pyobjc installed to automate desktop tasks on macOS. These dependencies enable input emulation for mouse and keyboard control and provide the necessary Python bindings for the local macOS Vision framework integration.

Related Skills

Explore More Software Engineering Skills→
aaronbassettaaronbassett

claude-bell

Send macOS notifications and collect user input via the cb CLI.

Community
Basic
Brilly-BohyunBrilly-Bohyun

yuanbao

Send @mention notifications, query group information, and message members.

Community
Basic
tilework-techtilework-tech
149

Taking and Analyzing Screenshots

Capture screen content and analyze images to diagnose UI bugs.

Official
Intermediate
software-mansionsoftware-mansion
1.7k

react-native-executorch

Integrate on-device AI models into React Native applications using ExecuTorch.

Official
Advanced