skills.rest
pattobrien
pattobrienCOMMUNITY
Explore all skills by this author →
GitHub

peekaboo

Automates macOS desktop UI inspection and control via the Peekaboo CLI.

2|Updated Oct 16, 2021
Search Tags:#automation#accessibility#cli#macos#ui testing#peekaboo#desktop control
One-click install
npx skills add https://github.com/pattobrien/dotfiles --skill peekaboo-pattobrien
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: peekaboo
Source: https://github.com/pattobrien/dotfiles/tree/main/.agents/skills/peekaboo
Command: npx skills add https://github.com/pattobrien/dotfiles --skill peekaboo-pattobrien

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Automating macOS desktop applications requires knowing the current UI state, element identifiers, and correct input paths, which are hard to obtain and keep stable across sessions. This Skill provides workflows for capturing UI snapshots, clicking and typing through accessibility or synthetic input paths, and validating changes in the Peekaboo repository. ## Core Features & Use Cases - UI Inspection: Capture live UI maps with peekaboo see --json to get element IDs, bounds, and snapshot IDs for stable interactions. - Desktop Control: Click elements, type text, launch apps, list windows, and manage clipboard and permissions through the CLI. - Input Path Testing: Distinguish UIAX/action-based input from synthetic CAEvent/CGEvent input using --input-strategy overrides and perform-action smoke tests. - Use Case: Automate a Calculator workflow by capturing its UI snapshot, extracting element IDs for buttons, and pressing them via accessibility actions without moving the pointer. ## Quick Start Ask the agent to run peekaboo see with JSON output on a target app, then click a discovered element using its snapshot ID.

Frequently Asked Questions about peekaboo

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I click a UI element in a macOS app from the command line?▼

Run peekaboo see --app <AppName> --json to capture element IDs and a snapshot ID, then run peekaboo click --on <element_id> --snapshot <id>. Using snapshot-backed element IDs is more stable than raw screen coordinates.

What is the difference between UIAX actions and synthetic input events?▼

UIAX actions invoke accessibility APIs like AXPress directly on elements, while synthetic input delivers pointer and keyboard events at coordinates. Use --input-strategy actionOnly or synthOnly to test each path independently.

Why does peekaboo fail to capture or control an app?▼

Most capture and control failures come from missing macOS permissions. Run peekaboo permissions status before assuming a CLI bug, and recapture with peekaboo see --json if the target UI has changed.

Can I use peekaboo output with other tools or scripts?▼

Yes, most commands support a --json flag so other tools can parse results. You can pipe output to utilities like ruby or jq to extract snapshot IDs, element bounds, and labels.

When should I use coordinates instead of element IDs for clicking?▼

Use coordinates only when accessibility metadata is unavailable for the target element. Element IDs from peekaboo see are preferred because they survive UI changes, and coordinates cannot use the actionOnly input strategy.

Related Skills

Explore More Software Engineering Skills→
letta-ailetta-ai
327

peekaboo

Automate macOS UI interactions and testing via the Peekaboo CLI.

Official
Advanced
nebinfranebinfra

neb-test-agent

Automate end-to-end agent capability validation across filesystem, git, helm, and platform APIs.

Official
Advanced
rmansrmans

scaffold-review-art

Review generated art assets for visual consistency and coverage gaps.

Community
Intermediate
iBz-04iBz-04
63

automating-mac-apps

Automates macOS apps via Apple Events using AppleScript discovery and JXA/PyXA with JSON output for pipelines.

Community
Intermediate