use-my-phone

Automate iPhone UI taps, swipes, and OCR data extraction via WebDriverAgent.

27|9|Updated Jan 25, 2026

Search Tags:#automation #ocr #scripting #touch #iphone #wda

Ready to run

npx skills add https://github.com/Leoyishou/personal-ai-company --skill use-my-phone

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: use-my-phone
Source: https://github.com/Leoyishou/personal-ai-company/tree/main/claude-global/skills/use-my-phone
Command: npx skills add https://github.com/Leoyishou/personal-ai-company --skill use-my-phone

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automate real iPhone UI interactions via WebDriverAgent to control apps on a physical device, enabling repeatable automation and testing.

Core Features & Use Cases

Automate taps, swipes, text input, and navigation on real iPhone apps using WDA REST API.
Capture screenshots and extract text via OCR for validation, auditing, and reporting.
Use cases include automated app demos, QA validation, and repetitive task automation across iOS apps.

Quick Start

Launch the WDA client and start a session with the target app bundle ID.

Frequently Asked Questions about use-my-phone

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate taps and swipes on a real iPhone?▼

Automate taps and swipes on a real iPhone by sending commands to the WebDriverAgent REST API. This skill uses Python snippets to perform UI interactions and automate repetitive tasks across installed iOS apps.

Can I extract text from iPhone apps for validation using OCR?▼

Extract text from iPhone apps using OCR by capturing screenshots via WebDriverAgent. The skill uses OCR to extract on-device data, enabling automated validation, auditing, and reporting of iOS app content.

What do I need to start automating iOS apps with WebDriverAgent?▼

To start automating iOS apps with WebDriverAgent, you need a running WDA service, a connected iPhone UDID, and optional port-forwarding setup. The skill reads the WDA REST API to execute Python automation snippets.

How does WebDriverAgent control installed apps on an iPhone?▼

WebDriverAgent controls installed apps by starting a session with the target app bundle ID and reading the WDA REST API. It enables automated app demos and QA validation by performing UI interactions like text input and navigation.

What are the limitations of automating iPhone UI interactions via WDA?▼

Automating iPhone UI interactions via WDA requires a continuously running WDA service and a physically connected device UDID. Automation relies on screenshot OCR for data extraction, which may be limited by on-screen rendering complexities.