desktop-find-element

Locate desktop UI elements via UIAutomation or vision recognition.

2.0k|267|Updated Jan 30, 2026
One-click install
npx skills add https://github.com/openakita/openakita --skill desktop-find-element
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: desktop-find-element
Source: https://github.com/openakita/openakita/tree/main/skills/system/desktop-find-element
Command: npx skills add https://github.com/openakita/openakita --skill desktop-find-element

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps you precisely locate and identify user interface elements on your desktop, enabling automated interactions with applications.

Core Features & Use Cases

  • Element Identification: Find buttons, menus, icons, and other UI components by their properties or visual appearance.
  • Positioning: Get the exact coordinates and dimensions of elements for accurate interaction.
  • UI State Verification: Confirm the presence and state of specific UI elements before proceeding with actions.
  • Use Case: Automate the process of clicking a "Save" button in any application by first finding its exact location on the screen.

Quick Start

Find the 'Settings' button on the screen.

Frequently Asked Questions about desktop-find-element

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I locate a UI element on my desktop for automation?

To locate a UI element for desktop automation, you can identify it using properties like its name and ID, or use vision recognition to find it by visual appearance. This allows scripts to target specific buttons or menus accurately.

Can I find a desktop UI element using a natural language description?

Yes, you can find a desktop UI element using a natural language description. The Skill supports identifying components like buttons and icons by interpreting text descriptions, alongside specifying window titles to narrow down the search scope.

What is the best way to get exact coordinates for UI testing?

The best way to get exact coordinates for UI testing is by locating the element through UIAutomation or vision recognition. This process returns the precise coordinates and dimensions required for accurate automated interactions.

Does UIAutomation work for verifying the state of an application window?

Yes, UIAutomation works for verifying the state of an application window. You can confirm the presence and specific state of UI elements within a targeted window before proceeding with further automation actions.

How to identify a button on the screen without knowing its ID?

You can identify a button on the screen without knowing its ID by utilizing vision recognition or providing a natural language description. This allows the system to locate the UI component based on its visual appearance and context.

What are the limitations of vision recognition for desktop automation?

A limitation of vision recognition for desktop automation is that it relies on visual appearance, which may fail if the UI element is obscured, off-screen, or dynamically rendered without distinct visual cues, unlike locating elements by name or ID.