app-control

Name: app-control
Availability: InStock
Author: cerisonautomation

Drive a named macOS application via raw keyboard and mouse input.

Updated Aug 23, 2026

Search Tags:#automation #macos #emulators #ui control #raw input #accessibility bypass #proxy tools

Ready to run

npx skills add https://github.com/CerisonAutomation/clud --skill app-control-cerisonautomation

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: app-control
Source: https://github.com/CerisonAutomation/clud/tree/main/assistant/src/config/bundled-skills/app-control
Command: npx skills add https://github.com/CerisonAutomation/clud --skill app-control-cerisonautomation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the problem of controlling a specific macOS application when the macOS Accessibility (AX) tree is unhelpful or empty.

Core Features & Use Cases

Direct app control via proxy tools: Starts a session, observes window state, then drives keyboard, mouse, typing, and screenshots through the connected macOS client.
App-targeted, deterministic interaction patterns: Uses observe before acting, supports sequences, combos, and text input for reliable shortcut execution.
Use cases: Emulators, games, OpenGL canvases, and custom-rendered Electron apps where standard UI navigation fails.

Quick Start

Tell the assistant which app to control (by bundle ID) and that you want it to drive the app using raw input because the Accessibility tree is unhelpful.

Frequently Asked Questions about app-control

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate macOS apps when the Accessibility tree is empty or unhelpful?▼

You can automate macOS apps with empty Accessibility trees by driving them through raw keyboard and mouse input, bypassing the AX tree entirely. This approach uses window-relative coordinates to interact directly with custom-rendered interfaces.

Can I send raw input to control an emulator or OpenGL canvas on macOS?▼

Yes, you can control emulators, games, and OpenGL canvases on macOS using raw input. Because these environments render custom UI without standard elements, raw keyboard and mouse actions interact directly with the canvas.

What is the best way to automate custom-rendered Electron apps that standard UI navigation fails on?▼

Automating custom-rendered Electron apps requires using raw input proxy tools that observe the window state and then execute keyboard, mouse, and screenshot actions. This ensures deterministic interaction when standard UI navigation fails.

How do I start a raw input automation session for a named macOS application?▼

To start a raw input automation session, identify the target app by its bundle ID and initiate the proxy tools. You must observe the window state and allow settle timing before executing input actions to ensure reliable automation.

Why does raw input automation require observe and settle timing before acting?▼

Raw input automation requires observe and settle timing because it bypasses the Accessibility tree and targets window-relative coordinates. Observing the state before acting ensures sequences, combos, and text input execute reliably without standard UI event feedback.

When should I not use Accessibility-based automation and switch to raw input on macOS?▼

You should switch to raw input when Accessibility-based automation is unreliable, specifically for emulators, games, OpenGL canvases, and custom-rendered Electron apps. If the AX tree is empty or unhelpful, raw keyboard and mouse input is the necessary alternative.