What problem does it solve? Automating the Windows desktop requires knowing how to reliably open apps, read on-screen state, and interact with UI elements without guessing coordinates or keystrokes. This Skill provides the operational playbook for driving the Windows Computer Use MCP server, so the agent observes before acting and picks the most deterministic interaction path. ## Core Features & Use Cases - UIA-tree-first interaction: Read the foreground window's accessibility tree via get_window_state, then click, scroll, or drag by element_index with pixel-coordinate fallback. - Deterministic app launching: Discover AUMIDs with list_installed_apps and launch apps through the official shell:AppsFolder path, avoiding name-guessing across languages. - Interaction priority rules: Prefer type_text for text input, click for single-target activation, and press_key only for genuine keyboard-only shortcuts, with recipes for Explorer, Settings, browsers, terminals, Office, and VSCode. - Use Case: Ask the agent to open Notepad, type a paragraph, and save it to the desktop — it will propose a plan, wait for approval, launch the app by AUMID, type the text, and press Ctrl+S. ## Quick Start Ask the agent to open the Windows Calculator app and compute 123 plus 456 using the computer use tools.
