peekaboo

Name: peekaboo
Availability: InStock
Author: elizaOS

Automate macOS UI interactions via command-line interface.

19.0k|5.6k|Updated Jul 9, 2024

Search Tags:#accessibility #cli #macos #ui automation #screen capture #application control

Ready to run

npx skills add https://github.com/elizaOS/eliza --skill peekaboo-elizaos

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: peekaboo
Source: https://github.com/elizaOS/eliza/tree/main/packages/skills/skills/peekaboo
Command: npx skills add https://github.com/elizaOS/eliza --skill peekaboo-elizaos

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates interactions with the macOS graphical user interface, allowing for programmatic control of applications, windows, and UI elements.

Core Features & Use Cases

UI Element Interaction: Click, type, drag, and scroll on specific UI elements identified by their properties.
Screen Capture & Analysis: Capture screenshots, record screen activity, and analyze visual content.
Application & Window Management: Launch, quit, focus, and manipulate application windows.
Use Case: Automate the process of filling out a complex web form within a desktop application, including handling dialog boxes and ensuring correct input fields are targeted.

Quick Start

Run peekaboo permissions to check necessary macOS permissions for UI automation.

Frequently Asked Questions about peekaboo

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate macOS UI interactions from the command line?▼

macOS UI automation from the command line is achieved by programmatically controlling applications, windows, menus, and UI elements to simulate input and manage system dialogs. This allows you to script complex desktop workflows without manual interaction.

What macOS permissions are required for UI automation and screen capture?▼

UI automation and screen capture on macOS require explicit accessibility and screen recording permissions. You can check if your system is configured correctly for these automation tasks by running the permissions verification command.

Can I capture screenshots and analyze visual content programmatically on macOS?▼

Yes, you can capture screenshots and record screen activity programmatically. The captured visual content can then be analyzed to verify UI states or extract information during automated application control workflows.

How do I target specific UI elements when automating a desktop application?▼

To target specific UI elements during desktop application automation, you identify them by their unique properties. This ensures that input simulation like clicking, typing, and dragging is applied to the correct fields and controls.

Does this UI automation approach work for handling system dialog boxes?▼

Yes, system dialog management is a core feature of this UI automation approach. It allows scripts to programmatically interact with and dismiss or respond to system dialogs that appear during automated workflows.