weavgui

What problem does it solve?

Automates desktop graphical user interface tasks by coordinating screenshot capture, mouse movements, clicks, and keystrokes, enabling precise interaction with on-screen elements.

Core Features & Use Cases

• Automated UI interactions: Perform clicks, keystrokes, and mouse movements based on visual cues and constant feedback.
• Screen content analysis: Capture and interpret screen images to locate UI components or verify actions.
• Use Case: Automate repetitive tasks like filling forms, navigating applications, or clicking buttons on the desktop by visually identifying targets and executing actions precisely.
• Technical scope: Integrates with vision-enabled LLMs and supports normalized coordinate-based control for platform-independent automation.

Quick Start

Use the weavgui CLI to capture a screen, move the mouse to the target position, then click, following the iterative verification process based on auto-captured screenshots.