skills.rest
davidrrowley
davidrrowleyCOMMUNITY
Explore all skills by this author →
Website𝕏@davidrrowleyGitHub

azure-ai-voicelive-ts

Stream bidirectional PCM16 voice audio in Node.js and browser environments.

Updated Apr 12, 2026
Search Tags:#typescript#azure#voice#realtime#websocket#function-calling#audio-streaming
One-click install
npx skills add https://github.com/davidrrowley/CortexYouV3 --skill azure-ai-voicelive-ts
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: azure-ai-voicelive-ts
Source: https://github.com/davidrrowley/CortexYouV3/tree/main/.agents/skills/azure-ai-voicelive-ts
Command: npx skills add https://github.com/davidrrowley/CortexYouV3 --skill azure-ai-voicelive-ts

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill removes the friction and complexity of implementing bidirectional real-time voice interactions by documenting session lifecycle, audio capture/playback, event subscription, turn detection, and function-calling patterns for TypeScript and JavaScript environments.

Core Features & Use Cases

  • Bidirectional audio streaming: Patterns for capturing microphone PCM16, sending audio to the VoiceLiveSession, and playing back streaming assistant audio.
  • Event-driven session management: Subscription-based handlers for connection lifecycle, transcription deltas, response text/audio deltas, error handling, and server events.
  • Function calling and tooling: Define JSON-schema tools in session config, handle streamed function arguments, return structured function outputs, and coordinate parallel tool executions.
  • Browser and Node support: Guidance for AudioWorklet-based capture/playback in browsers, bundler considerations, and Node-compatible client usage.

Quick Start

Create a VoiceLiveClient with an appropriate credential, start a realtime session for a realtime-capable model, update session modalities and audio formats to pcm16, subscribe to response events, and stream microphone PCM16 audio to the session while handling function calls and playback.

Frequently Asked Questions about azure-ai-voicelive-ts

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a real-time voice AI assistant with bidirectional audio streaming in TypeScript?▼

Real-time bidirectional voice streaming in TypeScript requires WebSocket session management, PCM16 audio capture, and playback. You configure a realtime session, subscribe to response events, and stream microphone audio while handling function calls and playback.

Can I use AudioWorklet for PCM16 audio capture and playback in a browser-based voice assistant?▼

AudioWorklet-based capture and playback is supported for browser environments. The pattern allows capturing microphone PCM16 audio, sending it to the voice session, and playing back streaming assistant audio directly within web applications.

How does function calling work in a real-time voice AI session?▼

Function calling in a real-time voice session involves defining JSON-schema tools in the session config, handling streamed function arguments, returning structured outputs, and coordinating parallel tool executions during the conversation flow.

What's the best way to manage WebSocket session lifecycle and event subscriptions for a conversational voice AI?▼

WebSocket session management uses event-driven subscription handlers for connection lifecycle, transcription deltas, response text/audio deltas, error handling, and server events to maintain bidirectional voice communication.

Does this real-time voice streaming approach work in Node.js as well as browsers?▼

Real-time voice streaming supports both Node.js and browser environments. Browser usage relies on AudioWorklet with bundler considerations, while Node-compatible client usage is supported for building speech-to-speech pipelines.

How do I configure turn detection for a speech-to-speech pipeline using WebSocket?▼

Turn detection configuration is set within the realtime session alongside modalities and audio formats. Configuring PCM16 formats and turn detection enables proper conversational flow handling for speech-to-speech pipelines.

Related Skills

Explore More Software Engineering Skills→
JuanMS20JuanMS20

github-auth

Configure GitHub authentication for Hermes Agent using git, gh CLI, or SSH credentials.

Community
Intermediate
kaivyykaivyy
20

Voice Skill

Transcribe audio files with OpenAI Whisper and synthesize speech from text.

Community
Intermediate
ruskibeatsruskibeats

extract-command-router-from-tool-handler

Extract command routing logic from TypeScript tool handlers into dedicated router modules.

Community
Intermediate
davidrrowleydavidrrowley

azure-ai-voicelive-dotnet

Enable bidirectional AI voice sessions in .NET using WebSockets and Azure authentication.

Community
Intermediate