skills.rest
openaeon
openaeonCOMMUNITY
Explore all skills by this author →
GitHub

peekaboo

Automate macOS desktop UI capture and interaction via the Peekaboo CLI.

87|20|Updated Mar 9, 2026
Search Tags:#accessibility#ui testing#macos automation#agent runtime#snapshot capture#synthetic input#desktop permissions
One-click install
npx skills add https://github.com/openaeon/OpenAEON --skill peekaboo-openaeon
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: peekaboo
Source: https://github.com/openaeon/OpenAEON/tree/main/skills/peekaboo
Command: npx skills add https://github.com/openaeon/OpenAEON --skill peekaboo-openaeon

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Peekaboo reduces the pain of unreliable UI automation on macOS by capturing the current UI state and executing interactions with accurate element IDs, snapshots, and correct accessibility or synthetic input strategies.

Core Features & Use Cases

  • UI capture and element discovery: Use live inspection to retrieve UIAX element IDs and snapshot IDs before acting, making interactions auditable and repeatable.
  • Reliable clicking and typing: Choose between UIAX/action execution and synthetic input, including fallbacks when focus or accessibility metadata behaves differently across apps.
  • Permissions-aware automation: Validate automation permissions up front to quickly distinguish user/setup issues from true CLI or target-app behavior.
  • Use Case: You need to regression-test or perform repetitive workflows in Calculator or another desktop app; you can verify permissions, capture a snapshot, click and type using discovered elements, and retry safely after UI changes.

Quick Start

Run peekaboo permissions status and then use peekaboo see --app Calculator --json to capture a snapshot before you perform any clicks or typing.

Frequently Asked Questions about peekaboo

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS UI interactions reliably when accessibility metadata behaves differently across apps?▼

Automate macOS UI interactions reliably by capturing a fresh snapshot to retrieve UIAX element IDs, then executing clicks and typing using actionOnly or synthOnly input strategies. This handles varying accessibility metadata and focus issues across apps by validating permissions and using fresh element IDs before acting.

What's the best way to capture macOS desktop UI elements for automated testing?▼

Capture macOS desktop UI elements for automated testing by using the see command with a target app to retrieve current UIAX element IDs and snapshot IDs. This produces parseable JSON results containing fresh element mappings, enabling auditable and repeatable interactions for regression testing workflows.

Do I need to validate macOS automation permissions before running desktop UI scripts?▼

Yes, you need to validate macOS automation permissions before running desktop UI scripts to quickly distinguish user setup issues from true target-app behavior. Run permissions status checks upfront to ensure the environment has the required accessibility and automation access before executing UI interactions.

How do I handle synthetic input fallback when UIAX actions fail during macOS automation?▼

Handle synthetic input fallback during macOS automation by selecting the synthOnly input strategy when UIAX actions fail or focus behaves unpredictably. This forces synthetic input execution, bypassing unreliable accessibility metadata to ensure typing and clicking actions complete successfully across varying desktop permission states.

Why does my macOS UI automation fail after the application window changes state?▼

macOS UI automation fails after application window changes because previously captured element IDs and snapshot IDs become stale. You must use the see command to obtain fresh element mappings before retrying actions, ensuring interactions target valid current UI state rather than outdated accessibility metadata.

Can I generate parseable JSON output from macOS UI screenshots for downstream agent processing?▼

Yes, you can generate parseable JSON output from macOS UI screenshots by executing capture and interaction commands with the JSON flag. This retrieves UIAX element IDs, snapshot IDs, and action results in a structured format, enabling downstream agents to process UI state and automate subsequent workflow steps reliably.

Related Skills

Explore More Software Engineering Skills→
steipetesteipete
385.7k

peekaboo

Capture and control macOS UI elements via CLI commands.

Community
Advanced
ohdaveedohdaveed

playwright-cli

Automate browser interactions and capture page state with Playwright CLI.

Community
Advanced
jluo41jluo41

haipipe-nn-0-overview

Outline the 4-layer NN pipeline architecture and YAML model templates.

Community
Advanced
autonomy-cloudautonomy-cloud
2

macos-automation

Automate native macOS app interactions using AppleScript via osascript.

Official
Intermediate