What problem does it solve? Characters with voice-over dialogue look lifeless when their mouths don't move with the words. This Skill turns an audio file into a replayable, sample-accurate lip sync animation by extracting phonemes, mapping them to visemes, and baking BlendShape tracks into the engine's Animation system. ## Core Features & Use Cases - Phoneme-to-viseme pipeline: Run Rhubarb Lip Sync (or a Whisper-phoneme cloud fallback) on VO audio, then bake the mouth cues into an Animation resource with one BlendShape track per ARKit viseme. - AnimationTree integration: Wire the baked clip into a OneShot node so lipsync fires in sync with AudioStreamPlayer3D playback from your dialogue system. - Emotional expression layer: Add authored BlendShape expressions (smile, surprise, brow raises) on an additive layer orthogonal to lipsync, driven by gameplay events. - Use Case: An NPC says "Welcome to the village, traveler." Generate the line with text-to-speech, extract mouth cues with Rhubarb, bake the viseme animation, and the character's mouth moves in sync with the audio. ## Quick Start Ask your agent to make the NPC's mouth move with a voice line by extracting phonemes from the audio and baking a viseme BlendShape animation onto the character's head mesh.
