skills.rest
BanmaXMBanmaXMCommunity·1 Agent Skills Included

operate-ui-by-screenshot

Safe screenshot-based control of desktop apps and websites

Operates desktop apps and websites through screenshots, DOM inspection, UI Automation, and safe coordinate clicks. Eliminates blind clicking, stale page states, and typing errors by verifying every action with screenshots and clipboard paste. Stops automatically at passwords, payments, and destructive actions, and saves reusable operation profiles for new apps and sites.
53|0
↗
npx skills add BanmaXM/operate-ui-by-screenshot --all -g -y

All Skills in This Repository (1)

Pure Emerald Level Indicators
📦 In Repo
BanmaXMBanmaXM

operate-ui-by-screenshot

Operate desktop apps and websites with visible state checks and verification screenshots.

Community
Advanced

Frequently Asked Questions

FAQPage Schema
How to install operate-ui-by-screenshot?▼

Run `npx skills add BanmaXM/operate-ui-by-screenshot --all -g -y` in your terminal to install the skill globally.

How to control desktop apps with screenshots?▼

The skill captures the screen, confirms the target is visible, then clicks coordinates or pastes text, verifying each step with a follow-up screenshot.

Does it prefer APIs over clicking buttons?▼

Yes. It always tries APIs, CLIs, DOM inspection, or UI Automation first, and only falls back to screenshot-based clicking when no structured interface exists.

Is it safe for logins and payments?▼

Yes. It automatically stops at passwords, 2FA, CAPTCHA, payments, and destructive actions, waiting for your explicit confirmation.

Which apps and sites are supported?▼

It ships with ready profiles for Edge, ChatGPT, GitHub, Kaggle, Luogu, and Adobe Premiere Pro, and can learn new apps by saving reusable profiles.

Related Repositories in Software Engineering

View All in Software Engineering→
openclawopenclaw
388.4k

openclaw

Run a personal AI assistant across your devices and chat apps

Official📦 120 Skills
obraobra
280.0k

superpowers

Gives coding agents a disciplined workflow from idea to merged code

Community📦 15 Skills
facebookfacebook
248.4k

react

AI agent skills for building, testing, and porting React core

Official📦 14 Skills