skills.rest
Castrozan
CastrozanCOMMUNITY
Explore all skills by this author →
GitHub

avatar

Orchestrate VTuber avatar lip-sync, expressions, and audio routing via WebSocket API.

3|Updated Jul 17, 2024
Search Tags:#tts#websocket#expressions#renderer#lip-sync#avatar-control#vtuber
One-click install
npx skills add https://github.com/Castrozan/.dotfiles --skill avatar
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: avatar
Source: https://github.com/Castrozan/.dotfiles/tree/main/agents/skills/avatar
Command: npx skills add https://github.com/Castrozan/.dotfiles --skill avatar

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires edge-tts, ffmpeg, pactl, xdotool, pw-link, curl, node, and includes scripts (resource) components.

What problem does it solve?

This Skill automates the orchestration of lip-sync, expressive changes, and audio routing for a VTuber avatar, reducing manual setup and latency during performances and troubleshooting.

Core Features & Use Cases

  • Lip-sync driven avatar animation and mouth movement synchronized with spoken text.
  • Real-time expression control and idle behavior coordination to reflect mood.
  • Renderer synchronization via a WebSocket API and HTTP audio endpoints, enabling live demos, Meet calls, and streams.
  • Use Case: During a live stream, drive the avatar to speak with lip-sync, switch expressions on the fly, and route audio to both room speakers and the virtual mic.

Quick Start

  1. Ensure the Avatar system is installed and start it: start-avatar.sh
  2. Speak with the avatar: avatar-speak.sh "Hello world" neutral speakers
  3. Change expression: avatar-speak.sh "Hello!" happy speakers

Frequently Asked Questions about avatar

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate VTuber avatar lip sync with text to speech?▼

Automating VTuber avatar lip sync uses edge-tts for speech generation and FFmpeg for playback, synchronizing mouth movements to spoken text via a WebSocket API and HTTP audio endpoints.

Can I control VTuber avatar expressions in real time during a live stream?▼

Yes, real time expression control allows switching avatar moods on the fly, coordinating idle behavior and expressive changes during live streams or Meet calls using the avatar control server.

Do I need a local avatar renderer to use automated lip sync and audio routing?▼

Yes, a local avatar renderer and control server are required to process WebSocket API commands and HTTP audio endpoints for lip sync animation and audio routing.

How do I route VTuber text to speech audio to both speakers and a virtual mic?▼

Audio routing directs edge-tts generated speech through FFmpeg playback to room speakers and a virtual mic simultaneously, enabling consistent audio output for live streams and demos.

What dependencies are required to set up VTuber avatar control?▼

Setting up VTuber avatar control requires edge-tts for speech, FFmpeg for playback, node for the control server, plus pactl, xdotool, and pw-link for audio routing and window automation.

Why is my VTuber avatar lip sync not working with the WebSocket API?▼

Lip sync failures often stem from the local avatar renderer not running, the control server being offline, or incorrect WebSocket API connection settings preventing expression and mouth movement synchronization.

Related Skills

Explore More Software Engineering Skills→
armanisadeghiarmanisadeghi
2

tts-audio-system

Integrate text-to-speech, speech-to-text, and audio playback via Cartesia, Groq PlayAI, and Whisper backends.

Community
Advanced
Pika-LabsPika-Labs
38

explainer

Create browser-driven explainer videos from URLs or GitHub repositories.

Official
Advanced
calesthiocalesthio
123

hedra-character-video

Generate consent-aware Hedra avatar videos from approved images and audio.

Community
Advanced
GroundMountCompanyGroundMountCompany

ai-talking-head

Generate AI talking head videos with lip-sync using multiple model backends.

Community
Advanced