Verified:GitHub

peekaboo

Automate macOS UI capture and interaction via snapshot inspection and JSON output.

Updated Feb 8, 2026

Search Tags:#screen capture #keyboard input #macos ui automation #peekaboo cli #element targeting #menu and window control #json scripting

Ready to run

npx skills add https://github.com/nomad3/openclaw-k8s --skill peekaboo-nomad3

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: peekaboo
Source: https://github.com/nomad3/openclaw-k8s/tree/main/package/skills/peekaboo
Command: npx skills add https://github.com/nomad3/openclaw-k8s --skill peekaboo-nomad3

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires peekaboo.

What problem does it solve?

It solves the problem of manually clicking, navigating, and scripting macOS UI interactions by letting you target on-screen elements precisely and drive the interface programmatically.

Core Features & Use Cases

Capture & Inspect UI: Capture screens/windows/regions, generate snapshot IDs, and optionally analyze/annotate results for reliable targeting.
Element-Driven Interaction: Click, type, scroll, drag, swipe, hotkey, and gesture across apps using element IDs/queries/coordinates with smart waits.
UI Automation Orchestration: Manage apps/windows/menus/spaces/dock/menubar, run scripted flows, and reuse snapshot caches for faster repeat runs.

Quick Start

Run peekaboo permissions, then use peekaboo see with annotation to identify a target and immediately follow it by clicking that target and typing your input into the focused UI.

Frequently Asked Questions about peekaboo

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate macOS UI interactions like clicking and typing programmatically?▼

macOS UI automation targets on-screen elements precisely and drives the interface programmatically. You capture screens, inspect element snapshots, and execute clicks, typing, and gestures using element IDs or coordinates.

Can I script macOS window and menu control using JSON output?▼

Yes, macOS UI automation supports command automation with JSON output for scripting. You can orchestrate apps, windows, menus, spaces, dock, and menubar while reusing snapshot caches for faster repeat runs.

What permissions do I need to capture screens and drive input on macOS?▼

macOS UI automation requires the Peekaboo CLI with macOS accessibility and screen recording permissions. You run the permissions command first to grant access before capturing or interacting with UI elements.

How do I identify and target specific UI elements for scripted interaction?▼

You identify target UI elements by capturing screens or windows with annotation enabled. This generates snapshot IDs that let you immediately click the identified target and type input into the focused UI.

Does this approach support smart waits and gesture automation across macOS apps?▼

Yes, UI automation supports smart waits alongside clicking, typing, scrolling, dragging, swiping, hotkeys, and gestures across apps. It targets elements using IDs, queries, or coordinates for reliable scripted flows.

What are the limitations of CLI-based macOS UI automation?▼

CLI-based macOS UI automation depends on granted accessibility and screen recording permissions. It requires snapshot inspection to target elements accurately, so dynamic or unidentified UI elements may pose targeting challenges.