skills.rest
wangqianCAI
wangqianCAICOMMUNITY
Explore all skills by this author →
GitHub

peekaboo

Automate macOS UI interactions via accessibility and screen recording APIs.

Updated Jun 18, 2026
Search Tags:#automation#accessibility#cli#macos#scripting#ui-testing
One-click install
npx skills add https://github.com/wangqianCAI/OBI --skill peekaboo-wangqiancai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: peekaboo
Source: https://github.com/wangqianCAI/OBI/tree/main/skills/peekaboo
Command: npx skills add https://github.com/wangqianCAI/OBI --skill peekaboo-wangqiancai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the challenge of automating legacy or non-API-driven macOS applications by providing a direct interface to inspect UI elements, capture screen states, and simulate human-like input.

Core Features & Use Cases

  • UI Inspection: Identify and target specific application windows, menus, and buttons using visual snapshots or element IDs.
  • Interaction Automation: Execute complex sequences like drag-and-drop, hotkey combinations, and text typing with human-like timing profiles.
  • Use Case: Automate repetitive data entry into a desktop application that lacks a public API by using the see command to locate fields and the type command to input data.

Quick Start

Use the peekaboo skill to capture the current screen state and click the login button on the active Safari window.

Frequently Asked Questions about peekaboo

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS UI interactions for desktop apps without an API?▼

macOS UI automation for non-API-driven applications is achieved by interfacing directly with system accessibility and screen recording APIs. This allows programmatic control over application windows, menus, and mouse or keyboard input to orchestrate cross-app workflows.

Do I need special permissions to run macOS UI automation scripts?▼

Yes, macOS UI automation requires appropriate system-level permissions for screen recording and accessibility. You must grant these permissions to execute scripts that capture screen states and simulate human-like input across desktop applications.

How does UI inspection work for automating legacy macOS applications?▼

UI inspection identifies and targets specific application windows, menus, and buttons using visual snapshots or element IDs. This enables direct interaction with legacy macOS applications by capturing screen states and simulating complex input sequences like drag-and-drop and hotkey combinations.

What is the best way to automate repetitive data entry into a macOS desktop application?▼

Automating repetitive data entry into a macOS application involves using UI inspection to locate input fields and automation commands to type data. This approach leverages system accessibility APIs to simulate human-like typing profiles for cross-app workflow orchestration.

Can I execute complex input sequences like drag-and-drop and hotkeys on macOS?▼

Yes, you can execute complex input sequences such as drag-and-drop, hotkey combinations, and text typing on macOS. The automation leverages system accessibility APIs to simulate these interactions with human-like timing profiles across various application windows.

Related Skills

Explore More Software Engineering Skills→
elizaOSelizaOS
19.0k

peekaboo

Automate macOS UI interactions via command-line interface.

Official
Advanced
zrongzrong
11

os-use

Automate cross-platform desktop tasks with screen capture and input simulation.

Community
Advanced
adoreseveradoresever
202

peekaboo

Automate macOS UI interactions via the Peekaboo CLI.

Community
Advanced
ghufronbagaskaraghufronbagaskara

computer-use

Automate desktop GUI applications via accessibility frameworks without disrupting user input focus.

Community
Advanced