skills.rest
bitwarden
bitwardenOFFICIAL
Explore all skills by this author →
Website𝕏@bitwardenGitHub

interacting-with-android-device

Captures Android UI state and automates device interactions via MCP tools backed by ADB.

9.3k|1.0k|Updated Apr 30, 2016
Search Tags:#mcp#ui testing#android#screenshot#adb#device automation#ui hierarchy
One-click install
npx skills add https://github.com/bitwarden/android --skill interacting-with-android-device
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: interacting-with-android-device
Source: https://github.com/bitwarden/android/tree/main/.claude/skills/interacting-with-android-device
Command: npx skills add https://github.com/bitwarden/android --skill interacting-with-android-device

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manually inspecting Android UI hierarchies, calculating tap coordinates, and verifying screen changes through raw ADB commands is slow and error-prone. This Skill provides structured MCP tools that capture UI state, find elements, detect obstructions, and verify interactions automatically.

Core Features & Use Cases

  • UI State Capture: Dump the view hierarchy as XML and take screenshots in one call for structural and visual verification against design mocks.
  • Element Interaction: Find elements by text or content-desc, tap them with automatic obstruction detection, input text, and navigate home, back, or the app drawer.
  • Obstruction Detection: Identify system overlays and in-app elements (FABs, dialogs, snackbars) that would intercept taps, with automatic coordinate adjustment.
  • Use Case: While testing a new settings screen, capture the UI hierarchy, tap through Settings to Software updates using element text, and verify each screen transition with screenshots.

Quick Start

Use the android-device tools to capture the current screen, tap the element labeled Settings, and verify the new screen appears.

Frequently Asked Questions about interacting-with-android-device

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I tap an element on an Android device by its text?▼

Use the tap_element tool with the element's text or content-desc value. It finds the element, detects obstructions, adjusts coordinates if needed, taps, and captures a screenshot in one call.

How to capture Android UI hierarchy and screenshots programmatically?▼

Use the capture tool, which saves view.xml containing the UI hierarchy and screen.png as a screenshot to the working directory. Read the XML to find element bounds, text, and resource IDs.

What happens when another element blocks the tap target on Android?▼

The find_element and tap_element tools detect obstructions from system overlays and in-app elements like dialogs or FABs. Coordinates are auto-adjusted to the largest unobstructed region, and the response reports the obstructor's identity and bounds.

When should I use raw ADB commands instead of the MCP tools?▼

Use raw ADB for gestures not covered by the tools, such as custom swipe actions via adb shell input swipe, or specific key events. The MCP tools are preferred for capture, find, tap, navigate, and text input because they return structured results.

Why is my Android device not detected by the tools?▼

Check the USB connection or emulator status, enable USB debugging in Developer Options, and accept the RSA key prompt on the device. Run adb devices to confirm the device is visible before retrying.

Related Skills

Explore More Software Engineering Skills→
BuilderIOBuilderIO
11

android-native

Build and deploy Android APKs to emulators and devices using Gradle and adb.

Official
Intermediate
everrunseverruns
42

ui-screenshots

Capture web page screenshots with agent-browser for visual verification.

Official
Intermediate
dinoudondinoudon

accessibility-audit

Audit web applications for WCAG 2.2 compliance across seven accessibility categories.

Community
Advanced
hungson175hungson175

android-app-testing

Automate Android UI tests with Appium and the Espresso driver.

Community
Advanced