windows-ui-automation

Simulate mouse movement, clicks, and keyboard input in Windows GUI applications.

Updated Apr 5, 2026
One-click install
npx skills add https://github.com/dingdyan/openclaw-workspace-v2 --skill windows-ui-automation-dingdyan
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: windows-ui-automation
Source: https://github.com/dingdyan/openclaw-workspace-v2/tree/main/skills/windows-ui-automation
Command: npx skills add https://github.com/dingdyan/openclaw-workspace-v2 --skill windows-ui-automation-dingdyan

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill solves the problem of manually performing repetitive Windows desktop UI actions by programmatically moving the mouse, sending keyboard input, and controlling window focus.

Core Features & Use Cases

  • Mouse automation: Move the cursor and perform left, right, or double clicks for interacting with non-web applications.
  • Keyboard automation: Type text and send special keys (e.g., Enter, Tab, Alt) to drive forms and dialogs.
  • Window interaction: Focus a target window by title so subsequent mouse/keyboard actions apply to the correct app.
  • Use case: Automate logging into a desktop tool by clicking the username field and typing credentials, then pressing Enter to advance.

Quick Start

Use the windows-ui-automation skill to move the mouse to coordinate (500, 500), click, then type a message and press Enter in the focused desktop window.

Frequently Asked Questions about windows-ui-automation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate Windows desktop UI clicks and keyboard input?

To automate Windows desktop UI interactions, you can use PowerShell scripts that simulate mouse movement, clicks, and keyboard input. This approach leverages WinForms SendKeys and user32 mouse_event for deterministic input injection into non-web applications.

How can I focus a specific desktop window before sending automated keyboard input?

You can focus a specific desktop window by passing the window title as a parameter to the automation script. This ensures the subsequent mouse clicks and SendKeys keyboard input apply to the correct active application context.

Can I use PowerShell to send special keys like Enter and Tab to Windows dialogs?

Yes, you can use PowerShell with WinForms SendKeys to send special keys such as Enter, Tab, and Alt to Windows dialogs. You provide the specific key strings as parameters to the entry script to drive form submissions.

What is the best way to automate logging into a non-web desktop application?

Automating desktop logins involves using a PowerShell script to activate the target window by title, move the mouse to click the username field, type credentials via SendKeys, and press Enter to advance through the dialog.

Do I need any external dependencies to simulate mouse movement and clicks on Windows?

No external dependencies are required. The automation relies on built-in Windows capabilities, utilizing PowerShell scripts to execute user32 mouse_event for mouse control and WinForms SendKeys for keyboard input injection.