skills.rest
hafley66
hafley66COMMUNITY
Explore all skills by this author →
GitHub

screenpipe-architecture

Extract and index macOS on-screen text using AX and OCR with SQLite storage.

1|Updated Mar 15, 2026
Search Tags:#ocr#sqlite#text-extraction#ax#fts5#screenpipe#simhash
One-click install
npx skills add https://github.com/hafley66/claude-research --skill screenpipe-architecture
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: screenpipe-architecture
Source: https://github.com/hafley66/claude-research/tree/main/skills/screenpipe-architecture
Command: npx skills add https://github.com/hafley66/claude-research --skill screenpipe-architecture

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Screenpipe's production pattern for robust on-screen text capture, extraction, and indexing across macOS apps using AX + OCR with dedup and SQLite storage.

Core Features & Use Cases

  • AX tree walking to read focused window content and UI chrome where applicable.
  • Apple Vision OCR pipeline to extract text from screen captures with bounding boxes.
  • Hybrid routing between accessibility and OCR to maximize content coverage.
  • Event-driven capture for responsive updates and deduplicated storage in SQLite.
  • FTS5 indexing for fast queries across app name, window, and text content.
  • Use cases: real-time text capture for accessibility tooling, archiving screen content, and automated text extraction for research.

Quick Start

Run the screenpipe-architecture workflow on macOS to start capturing, extracting, and indexing on-screen text.

Frequently Asked Questions about screenpipe-architecture

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract on-screen text from macOS applications using accessibility and OCR?▼

On-screen text extraction on macOS uses a hybrid pipeline that walks AX trees for focused window content and applies Apple Vision OCR to screen captures. This hybrid routing maximizes content coverage by leveraging accessibility data first, then falling back to OCR for visual elements.

What is the best way to deduplicate captured screen text during continuous capture?▼

Deduplicating captured screen text is handled using SimHash during event-driven capture. This prevents redundant text from being stored repeatedly by comparing visual similarity hashes before committing the extracted content to the database.

Can I use SQLite FTS5 to index and query text extracted from screen captures?▼

SQLite FTS5 indexes extracted screen text to enable fast queries across app name, window, and text content. This allows you to perform full-text searches on the captured and deduplicated data stored locally on your macOS device.

Does this on-screen text extraction approach work for archiving content across multiple app windows?▼

On-screen text extraction works across multiple macOS apps and windows by using an event-driven capture loop. It systematically indexes content from different active applications, making it suitable for archiving screen content or feeding accessibility tooling.

When should I use Apple Vision OCR instead of AX tree walking for text capture?▼

Apple Vision OCR is used when AX tree walking cannot read focused window content or UI chrome effectively. The hybrid router automatically applies OCR to screen captures with bounding boxes to extract text from visual elements lacking accessibility labels.

Related Skills

Explore More Software Engineering Skills→
ldzhhxxldzhhxx
1

qmd

Index and search markdown documents with BM25, vector search, and LLM reranking.

Community
Intermediate
IOchairIOchair
66

sql-manything

Indexes source code into SQLite and extracts snippets via SQL+FTS5 searches.

Community
Advanced
cornmanwtfcornmanwtf

cdc-pipeline

Implement change data capture and event-driven synchronization patterns for data integration.

Community
Advanced
sky770825sky770825

screen-vision

Extract screen text and locate element coordinates using macOS Vision Framework.

Community
Intermediate