skills.rest
softwarebyze
softwarebyzeCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

argent-device-interact

Interact with iOS simulators and Android emulators using normalized coordinates.

Updated May 7, 2026
Search Tags:#automation#mobile testing#keyboard input#simulator control#android adb#ios accessibility#ui gestures
One-click install
npx skills add https://github.com/softwarebyze/Backgammon-Mastermind --skill argent-device-interact
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: argent-device-interact
Source: https://github.com/softwarebyze/Backgammon-Mastermind/tree/main/.agents/skills/argent-device-interact
Command: npx skills add https://github.com/softwarebyze/Backgammon-Mastermind --skill argent-device-interact

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill removes the trial-and-error of interacting with iOS simulators and Android emulators by providing a consistent set of touch, keyboard, navigation, and screenshot workflows across platforms.

Core Features & Use Cases

  • Unified device interaction: Tap, swipe/scroll, pinch/rotate, type, press hardware buttons, and rotate orientation using normalized coordinates (0.0–1.0).
  • App opening without brittle UI navigation: Launch apps by bundle ID or open URLs/schemes instead of tapping home-screen icons.
  • Reliable UI target discovery: Guide you to use accessibility discovery tools like describe/debugger-component-tree/native-describe-screen before resorting to screenshots.
  • Sequencing for multi-step actions: Batch steps with run-sequence for deterministic flows like scrolling multiple times or typing+submitting.
  • Platform caveat handling: Includes guidance for Android adb reverse/permissions and iOS describe limitations, plus fallbacks when accessibility inspection fails.

Quick Start

Use argent-device-interact after selecting a booted device with list-devices, then tap a UI element using gesture-tap with normalized x and y coordinates.

Frequently Asked Questions about argent-device-interact

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate touch and keyboard input on iOS simulators and Android emulators?▼

You can automate mobile testing across iOS and Android by mapping touch, keyboard, and hardware button actions onto a unified tool interface using normalized coordinates ranging from 0.0 to 1.0 for consistent cross-platform input.

What is the best way to launch an app in a mobile simulator without tapping UI icons?▼

The best way to launch apps without brittle UI navigation is to launch them directly by bundle ID or open specific URLs and schemes on the booted device, bypassing home-screen icon interactions entirely.

Can I batch multiple UI gestures and typing actions for automated testing?▼

Yes, you can batch multi-step actions like scrolling multiple times or typing and submitting text using the run-sequence tool, which executes deterministic interaction flows when intermediate results are not needed.

How do I find UI targets for gestures before executing automated taps?▼

Find UI targets for automated gestures by using accessibility discovery tools like describe, debugger-component-tree, or native-describe-screen before resorting to screenshots, with fallbacks included when accessibility inspection fails.

Do I need to boot a device before running UI automation sequences?▼

Yes, you must call list-devices and boot-device to select a ready device first, then use accessibility tools for target discovery before executing gesture and keyboard tools with correct platform-specific constraints.

What are the limitations of using normalized coordinates for Android adb and iOS accessibility?▼

Limitations include handling platform-specific constraints such as Android adb reverse and permissions configurations, plus working around iOS describe limitations by using fallback workflows when accessibility inspection fails.

Related Skills

Explore More Software Engineering Skills→
softwarebyzesoftwarebyze

argent-ios-simulator-setup

Boot an iOS simulator by UDID and verify gesture tool connectivity.

Community
Basic
sapdev-aisapdev-ai
7

sap-gui-probe

Record SAP GUI screen states and generate deterministic VBS replay scripts.

Official
Advanced
software-mansionsoftware-mansion
1.9k

argent-device-interact

Automate taps, gestures, typing, and app launches on iOS simulators and Android emulators.

Official
Intermediate
samttoo22-MewCatsamttoo22-MewCat
19

peekaboo

Automate macOS UI interactions and screen captures via command-line interface.

Community
Advanced