skills.rest
hafley66
hafley66COMMUNITY
Explore all skills by this author →
GitHub

macos-accessibility-rust

Read macOS AXUIElement trees to extract visible text and compute element bounds in Rust.

1|Updated Mar 15, 2026
Search Tags:#accessibility#permissions#macos#rust#text-extraction#axuielement#ui-tree-walking
One-click install
npx skills add https://github.com/hafley66/claude-research --skill macos-accessibility-rust
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: macos-accessibility-rust
Source: https://github.com/hafley66/claude-research/tree/main/skills/macos-accessibility-rust
Command: npx skills add https://github.com/hafley66/claude-research --skill macos-accessibility-rust

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

macOS applications expose their UI as AXUIElement trees. This Skill enables Rust tooling to traverse those trees, extract visible text, and determine screen-space bounds to power accessibility tooling and UI analysis.

Core Features & Use Cases

  • Tree walking of AXUIElement trees to locate nodes (e.g., AXStaticText) and extract their text.
  • Text extraction and bounding-box queries to map UI to screen coordinates for overlays or automation.
  • Guidance on the required Accessibility permission and how to operate in system-wide vs PID-scoped contexts.

Quick Start

Instantiate an AXUIElement, walk the tree, and collect text and positions from a target app.

Frequently Asked Questions about macos-accessibility-rust

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract visible text from macOS applications using Rust?▼

To extract visible text from macOS applications using Rust, you traverse AXUIElement trees to locate specific nodes like AXStaticText and use AXUIElementCopyAttributeValue to retrieve their text content. This process enables screen reader workflows and UI automation.

What is an AXUIElement tree and how does it help with UI discovery?▼

An AXUIElement tree is the hierarchical structure macOS applications expose to represent their UI components. Walking this accessibility tree allows you to discover nodes, extract visible text, and compute element bounds for screen-space mapping and UI analysis.

Do I need special permissions to read the accessibility tree in macOS?▼

Yes, reading the accessibility tree requires explicit Accessibility permissions in macOS. This Skill provides guidance on respecting the Accessibility permission model while operating in system-wide or PID-scoped contexts to query UI elements securely.

Can I compute element bounding boxes for UI automation using Rust?▼

Yes, you can compute element bounding boxes in Rust by querying AXUIElement attributes to determine screen-space bounds. This allows you to map UI elements to screen coordinates for overlays or automated UI interactions.

What's the difference between system-wide and PID-scoped accessibility queries?▼

System-wide accessibility queries target the entire macOS UI, while PID-scoped queries limit tree walking to a specific application process. This Skill demonstrates using AXUIElementGetPid and both query scopes for targeted text extraction.

Which Rust crates do I use to interact with macOS accessibility APIs?▼

You use the accessibility and accessibility-sys Rust crates to interact with macOS accessibility APIs. These crates provide the necessary bindings to call AXUIElement APIs for tree walking and attribute extraction.

Related Skills

Explore More Software Engineering Skills→
Probably-GroupProbably-Group
3

macos-accessibility

Inspect and control macOS UI elements via AXUIElement with permission checks.

Official
Advanced
smykla-skalskismykla-skalski
3

ocr-finder

Detects and extracts text from image files with bounding boxes and click coordinates using EasyOCR.

Official
Intermediate
Project-N-E-K-OProject-N-E-K-O
2.4k

3d-camera-interaction

Align mouse input with 3D model drag and zoom in Three.js.

Official
Advanced
wanhaoniuwanhaoniu
41

macos-use-desktop-control

Automate macOS desktop UI interactions by locating elements via visible text.

Community
Intermediate