skills.rest
elizaOS
elizaOSOFFICIAL
Explore all skills by this author →
Website𝕏@elizaOSGitHub

peekaboo

Automate macOS UI interactions via command-line interface.

19.0k|5.6k|Updated Jul 9, 2024
Search Tags:#accessibility#cli#macos#ui automation#screen capture#application control
One-click install
npx skills add https://github.com/elizaOS/eliza --skill peekaboo-elizaos
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: peekaboo
Source: https://github.com/elizaOS/eliza/tree/main/packages/skills/skills/peekaboo
Command: npx skills add https://github.com/elizaOS/eliza --skill peekaboo-elizaos

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates interactions with the macOS graphical user interface, allowing for programmatic control of applications, windows, and UI elements.

Core Features & Use Cases

  • UI Element Interaction: Click, type, drag, and scroll on specific UI elements identified by their properties.
  • Screen Capture & Analysis: Capture screenshots, record screen activity, and analyze visual content.
  • Application & Window Management: Launch, quit, focus, and manipulate application windows.
  • Use Case: Automate the process of filling out a complex web form within a desktop application, including handling dialog boxes and ensuring correct input fields are targeted.

Quick Start

Run peekaboo permissions to check necessary macOS permissions for UI automation.

Frequently Asked Questions about peekaboo

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate macOS UI interactions from the command line?▼

macOS UI automation from the command line is achieved by programmatically controlling applications, windows, menus, and UI elements to simulate input and manage system dialogs. This allows you to script complex desktop workflows without manual interaction.

What macOS permissions are required for UI automation and screen capture?▼

UI automation and screen capture on macOS require explicit accessibility and screen recording permissions. You can check if your system is configured correctly for these automation tasks by running the permissions verification command.

Can I capture screenshots and analyze visual content programmatically on macOS?▼

Yes, you can capture screenshots and record screen activity programmatically. The captured visual content can then be analyzed to verify UI states or extract information during automated application control workflows.

How do I target specific UI elements when automating a desktop application?▼

To target specific UI elements during desktop application automation, you identify them by their unique properties. This ensures that input simulation like clicking, typing, and dragging is applied to the correct fields and controls.

Does this UI automation approach work for handling system dialog boxes?▼

Yes, system dialog management is a core feature of this UI automation approach. It allows scripts to programmatically interact with and dismiss or respond to system dialogs that appear during automated workflows.

Related Skills

Explore More Software Engineering Skills→
wangqianCAIwangqianCAI

peekaboo

Automate macOS UI interactions via accessibility and screen recording APIs.

Community
Advanced
everrunseverruns
42

ui-screenshots

Capture web page screenshots with agent-browser for visual verification.

Official
Intermediate
balaraj74balaraj74

peekaboo

Automate macOS UI interactions with screen capture and input simulation.

Community
Advanced
badbsallyybadbsallyy

peekaboo

Automate macOS UI interactions and capture screen content via the Peekaboo CLI.

Community
Advanced