azure-ai-voicelive-ts

What problem does it solve?

This Skill removes the friction and complexity of implementing bidirectional real-time voice interactions by documenting session lifecycle, audio capture/playback, event subscription, turn detection, and function-calling patterns for TypeScript and JavaScript environments.

Core Features & Use Cases

• Bidirectional audio streaming: Patterns for capturing microphone PCM16, sending audio to the VoiceLiveSession, and playing back streaming assistant audio.
• Event-driven session management: Subscription-based handlers for connection lifecycle, transcription deltas, response text/audio deltas, error handling, and server events.
• Function calling and tooling: Define JSON-schema tools in session config, handle streamed function arguments, return structured function outputs, and coordinate parallel tool executions.
• Browser and Node support: Guidance for AudioWorklet-based capture/playback in browsers, bundler considerations, and Node-compatible client usage.

Quick Start

Create a VoiceLiveClient with an appropriate credential, start a realtime session for a realtime-capable model, update session modalities and audio formats to pcm16, subscribe to response events, and stream microphone PCM16 audio to the session while handling function calls and playback.