Convert desktop GUI interaction paths into reusable vision-mcp.yaml maps.
npx skills add https://github.com/Haruhiyuki/vision-mcp --skill vision-mcp
Please help me install this Agent Skill. Skill: vision-mcp Source: https://github.com/Haruhiyuki/vision-mcp/tree/main/skills/vision-mcp Command: npx skills add https://github.com/Haruhiyuki/vision-mcp --skill vision-mcp
Vision-MCP reduces the heavy, repetitive cost of desktop GUI tasks by turning each interaction path (screenshots, coordinate estimation, AX/OCR, and click sequences) into a reusable vision-mcp.yaml map that can be replayed with run_workflow. It enables agents to skip exhaustive visual exploration on subsequent runs, cutting initial setup time from minutes to seconds. The approach assumes the agent can perform visual operations and uses a modular map as an amortized automation layer rather than a replacement for existing computer-use skills.
Run vision-mcp workflow with an existing map to execute a desktop task end-to-end.
High-intent search queries and answers about installing and using this skill.
Desktop GUI automation can be sped up by converting interaction paths into a reusable YAML map. The map captures screenshots and click sequences, allowing subsequent workflow runs to skip visual exploration and execute instantly with near-zero setup time.
Cross-platform desktop automation maps support both macOS and Windows. The system includes platform-specific handling and safety checks to ensure that reusable interaction paths and click sequences operate reliably across different operating systems.
Handling UI changes in automated desktop workflows is done through patchable maps. When an interface updates, you can apply patches and repairs to the existing YAML map to keep the automation reliable without rebuilding the entire interaction path from scratch.
Building a desktop automation map uses guided exploration to capture visual operations like coordinate estimation and OCR. This task-driven process distills each interaction path into a structured YAML schema defining actions and state transitions for later reuse.
Desktop automation with reusable maps does not require visual operations on every run. After the first run captures the interaction path, subsequent tasks hit the workflow replay function, cutting the visual cost from minutes to seconds by using the amortized map.
Desktop GUI automation slows down on long workflows due to exhaustive visual exploration. Converting each interaction path into a reusable YAML map acts as an amortized automation layer, eliminating the repeated visual cost and speeding up cross-app execution.
hafley66Extract and index macOS on-screen text using AX and OCR with SQLite storage.
yt8766Execute written plans step by step with review checkpoints and verification.
coalesce-labsAutomate browser navigation, form filling, and UI testing with persisted sessions.
danielzmeowReduce visual intensity in UI/UX designs by desaturating colors and simplifying elements.
