skills.rest
testdriverai
testdriveraiOFFICIAL
Explore all skills by this author →
Website𝕏@testdriveraiGitHub

testdriver:elements

Locate UI elements by AI description and perform click, hover, and other interactions.

238|34|Updated Aug 25, 2024
Search Tags:#automation#testing#ai#interaction#find#element#ui-element
One-click install
npx skills add https://github.com/testdriverai/testdriverai --skill testdriver-elements
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: testdriver:elements
Source: https://github.com/testdriverai/testdriverai/tree/main/ai/skills/testdriver%3Aelements
Command: npx skills add https://github.com/testdriverai/testdriverai --skill testdriver-elements

SYSTEM DOCUMENTATION & REQUIREMENTS

## What problem does it solve? Locate and interact with UI elements on screen using AI to simplify automated UI testing, reducing flaky tests and speeding up test authoring.

## Core Features & Use Cases

  • Locate elements with natural language descriptions using testdriver.find().
  • Interact with located elements through an Element wrapper using methods like click, hover, doubleClick, rightClick, mouseDown, and mouseUp.
  • Access element metadata such as coordinates, size, text, label, and confidence for robust assertions.
  • Serialize element data for logging and cross-process sharing.

### Quick Start Use testdriver.find('submit button') to locate an element and then perform actions such as click.

Frequently Asked Questions about testdriver:elements

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find and interact with dynamic UI elements in automated tests?▼

You can locate dynamic UI elements in automated tests using AI natural language descriptions, then interact with them through methods like click, hover, and doubleClick to execute deterministic test flows.

Can I use natural language to locate elements on a dashboard or form?▼

Yes, you can use natural language to locate elements on dashboards and forms by calling the find function with descriptions, which returns an element wrapper for immediate interaction.

How do I automate mouse interactions like hovering or right-clicking in UI tests?▼

Automate mouse interactions in UI tests by using an element wrapper to call methods like hover, rightClick, mouseDown, and mouseUp on the located UI elements.

What metadata can I access from located UI elements for test assertions?▼

You can access element metadata including coordinates, size, text, label, and confidence scores from located UI elements to build robust assertions and verify test outcomes.

How do I serialize UI element data for logging and diagnostics?▼

Serialize UI element data for logging and cross-process sharing by using the built-in serialization features to safely capture coordinates and element metadata for diagnostic outputs.

Related Skills

Explore More Software Engineering Skills→
cxnaivecxnaive

findmy

Automate Apple's Find My app to track devices and AirTags via AppleScript and screen capture.

Community
Intermediate
mtmatteimtmattei
5

uno-app-test-assertions

Assert visual tree, element properties, and data bindings in Uno Platform UI tests.

Community
Intermediate
cqkxxccqkxxc

dispatching-parallel-agents

Coordinate parallel agents to investigate independent problems concurrently.

Community
Intermediate
web-infra-devweb-infra-dev
286

Desktop Computer Automation

Automate desktop environments on macOS, Windows, and Linux via visual input.

Official
Advanced