skills.rest
jaminitachi
jaminitachiCOMMUNITY
Explore all skills by this author →
GitHub

mac-control

Control macOS UI via Peekaboo CLI and AppleScript for automation.

1|2|Updated Feb 15, 2026
Search Tags:#automation#macOS#OCR#screenshot#AppleScript#UI control#Peekaboo
One-click install
npx skills add https://github.com/jaminitachi/SuperClaw --skill mac-control
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mac-control
Source: https://github.com/jaminitachi/SuperClaw/tree/main/skills/mac-control
Command: npx skills add https://github.com/jaminitachi/SuperClaw --skill mac-control

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill empowers AI agents to interact with and control the macOS graphical user interface, overcoming the limitations of terminal-only operations.

Core Features & Use Cases

  • UI Automation: Automate clicks, typing, and keyboard shortcuts within any macOS application.
  • Screen Interaction: Capture screenshots, read text from the screen (OCR), and manage application windows.
  • Use Case: An agent can be instructed to "fill out the system preferences form for display settings," and this skill will enable it to navigate the UI, input text, and click buttons to achieve the desired configuration.

Quick Start

Use the mac-control skill to take a screenshot of the current screen.

Frequently Asked Questions about mac-control

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS UI interactions and clicks with an AI agent?▼

AI agents can control macOS UI by using this skill to execute clicks, typing, and keyboard shortcuts through Peekaboo CLI and AppleScript. It translates agentic instructions into graphical interactions within any application.

Can I extract text from a macOS screenshot using OCR?▼

Yes, OCR text extraction from screenshots is supported natively. The skill captures screen images and reads the embedded text, enabling agents to parse visual information from applications that lack terminal APIs or accessibility hooks.

Does this approach work for managing application windows on macOS?▼

Yes, application window management is a core capability. Agents can dynamically interact with the graphical interface to resize, close, or arrange windows via AppleScript or JXA workflows during automated task execution.

What is the best way to fill out system preferences forms via UI control?▼

The best way to fill out system preferences is by instructing an agent to navigate the UI, input text, and click buttons. The skill handles the graphical navigation and deterministic form completion automatically.

Do I need Peekaboo CLI to perform dynamic macOS workflows?▼

Yes, Peekaboo CLI is required for graphical interactions like screenshots and OCR. For dynamic workflows, the skill also leverages AppleScript or JXA to execute tasks beyond terminal-only operations.

Related Skills

Explore More Software Engineering Skills→
geezerrrrgeezerrrr
117

peekaboo

Automate macOS UI tasks by commanding Peekaboo to inspect and interact with elements.

Community
Advanced
rvdbreemenrvdbreemen
199

pdf

Extracts and manipulates text, tables, images, and forms in PDF documents.

Community
Intermediate
PytaichukBohdanPytaichukBohdan

Fork Terminal Skill

Launch a separate terminal window and run a command via AppleScript or CMD.

Community
Intermediate
hwj123hwjhwj123hwj
4

wizard

Guide users through interactive bash procedures with configurable stages.

Community
Intermediate