Automate VRChat desktop control via CLI with OCR, template detection, and OSC actions.
npx skills add https://github.com/MLShukai/vrcpilot --skill vrcpilot-cli
Please help me install this Agent Skill. Skill: vrcpilot-cli Source: https://github.com/MLShukai/vrcpilot/tree/main/.claude/skills/vrcpilot-cli Command: npx skills add https://github.com/MLShukai/vrcpilot --skill vrcpilot-cli
This skill solves the problem of manually operating VRChat to capture frames, run OCR or template detection, and synthesize input through repetitive, error-prone steps.
It provides a deterministic CLI workflow for launching or targeting VRChat by PID, focusing the window for safe input, and performing screenshot capture that outputs YAML suitable for piping into OCR or detect stages. It supports end-to-end automation for VRChat window-local coordinate systems by combining screenshot → OCR (word-level results) → mouse movement, and by detecting image templates inside the same coordinate schema. It also enables integrated recording and streaming workflows via record (MP4/WAV files or self-describing MKV on stdout) and provides OSC command sending for in-world interactions.
Run vrcpilot screenshot and pipe its YAML into vrcpilot ocr to get recognized words in VRChat window-local coordinates for subsequent mouse actions.
High-intent search queries and answers about installing and using this skill.
Automating VRChat screenshots and OCR involves capturing frames via a CLI tool that outputs YAML, then piping that data into an OCR stage to extract word-level results in window-local coordinates for mouse automation. You can pipe the screenshot output directly into the OCR subcommand.
Yes, you can control VRChat using OSC commands and synthesized mouse input within a single CLI pipeline. The workflow targets the VRChat window by PID, focuses it for safe input execution, and sends OSC actions to trigger in-world interactions during the game loop.
VRChat CLI automation supports recording gameplay to MP4 and WAV files, as well as streaming self-describing MKV format directly to stdout. The stdout pipe support includes TTY safeguards to ensure proper stdin validation during the recording process.
The best way to click VRChat UI elements is by using image-template detection within the same window-local coordinate schema. This approach detects specific templates in captured screenshots and synthesizes mouse movements to click the identified UI targets deterministically.
Yes, YAML-based screenshot I/O is required to process VRChat screenshots for text recognition. The screenshot capture stage outputs YAML format, which is then piped or provided via the --screenshot flag directly into the OCR or detect subcommands.
VRChat automation fails with coordinate mismatches when window-local coordinate consistency is not maintained across the OCR, detect, and mouse synthesis stages. Ensuring the target window is properly focused by PID and applying a consistent coordinate schema prevents these alignment errors.
johnlindquistManage TypeScript and JavaScript script execution lifecycle in Script Kit.
KaiserWhoLearnsConvert media files between containers and codecs using FFmpeg.
caovinhphucAutomate browser navigation, form filling, data extraction, and screenshots via CLI.
kevinpbuckleyCapture Unreal Editor window, viewport, and asset screenshots via ScreenshotService API.
