skills.rest
Leoyishou
LeoyishouCOMMUNITY
Explore all skills by this author →
GitHub

use-my-phone

Automate iPhone UI taps, swipes, and OCR data extraction via WebDriverAgent.

27|9|Updated Jan 25, 2026
Search Tags:#automation#ocr#scripting#touch#iphone#wda
One-click install
npx skills add https://github.com/Leoyishou/personal-ai-company --skill use-my-phone
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: use-my-phone
Source: https://github.com/Leoyishou/personal-ai-company/tree/main/claude-global/skills/use-my-phone
Command: npx skills add https://github.com/Leoyishou/personal-ai-company --skill use-my-phone

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automate real iPhone UI interactions via WebDriverAgent to control apps on a physical device, enabling repeatable automation and testing.

Core Features & Use Cases

  • Automate taps, swipes, text input, and navigation on real iPhone apps using WDA REST API.
  • Capture screenshots and extract text via OCR for validation, auditing, and reporting.
  • Use cases include automated app demos, QA validation, and repetitive task automation across iOS apps.

Quick Start

Launch the WDA client and start a session with the target app bundle ID.

Frequently Asked Questions about use-my-phone

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate taps and swipes on a real iPhone?▼

Automate taps and swipes on a real iPhone by sending commands to the WebDriverAgent REST API. This skill uses Python snippets to perform UI interactions and automate repetitive tasks across installed iOS apps.

Can I extract text from iPhone apps for validation using OCR?▼

Extract text from iPhone apps using OCR by capturing screenshots via WebDriverAgent. The skill uses OCR to extract on-device data, enabling automated validation, auditing, and reporting of iOS app content.

What do I need to start automating iOS apps with WebDriverAgent?▼

To start automating iOS apps with WebDriverAgent, you need a running WDA service, a connected iPhone UDID, and optional port-forwarding setup. The skill reads the WDA REST API to execute Python automation snippets.

How does WebDriverAgent control installed apps on an iPhone?▼

WebDriverAgent controls installed apps by starting a session with the target app bundle ID and reading the WDA REST API. It enables automated app demos and QA validation by performing UI interactions like text input and navigation.

What are the limitations of automating iPhone UI interactions via WDA?▼

Automating iPhone UI interactions via WDA requires a continuously running WDA service and a physically connected device UDID. Automation relies on screenshot OCR for data extraction, which may be limited by on-screen rendering complexities.

Related Skills

Explore More Software Engineering Skills→
woonetworkwoonetwork

WOOFi Earn Summary Skill

Query the WOOFi API to rank cross-chain Supercharger vault APRs.

Official
Basic
alibabaalibaba
10.5k

pdf-extractor

Extract text, tables, and metadata from PDF documents into JSON.

Official
Intermediate
garrytangarrytan
127.1k

ios-qa

Automate iOS QA on real devices with vision-driven testing for SwiftUI apps.

Community
Advanced
ramunasnognysramunasnognys
1

mobile-testing-playwright

Automate iOS and Android mobile testing with Playwright MCP for UI and interaction issues.

Community
Intermediate