skills.rest
GoogleCloudPlatform
GoogleCloudPlatformOFFICIAL
Explore all skills by this author →
WebsiteGitHub

liveapi-service

Generates a LiveAPI websocket client service class in the user's chosen programming language.

783|307|Updated May 27, 2021
Search Tags:#websocket#vertex ai#client generation#session resumption#gemini live api#Multimodal#realtime streaming
One-click install
npx skills add https://github.com/GoogleCloudPlatform/vertex-ai-samples --skill liveapi-service
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: liveapi-service
Source: https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/skills/genai-sdk/references/live_api
Command: npx skills add https://github.com/GoogleCloudPlatform/vertex-ai-samples --skill liveapi-service

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Building a client for the Gemini Live API requires correctly implementing the bidirectional WebSocket protocol, session resumption, bearer token refresh, and proto message handling, which is error-prone when done from scratch.

Core Features & Use Cases

  • Client Service Generation: Scaffolds a LiveAPI service class in any programming language that connects to the Gemini LiveAPI websocket endpoint and exposes send/receive functions for ClientMessage and ServerMessage protos.
  • Session & Auth Handling: Implements session setup and resumption, bearer token refresh for Gemini Enterprise, and API key support for the Gemini Developer API.
  • Full-Stack Test UI: Produces a deployable frontend and backend service for testing live audio, video, and text interactions with the model, including interruption and transcription handling.
  • Use Case: Ask the Skill to build a Python LiveAPI client for your project, then use the generated UI to stream microphone audio and camera frames to Gemini and hear time-aligned audio responses with transcriptions.

Quick Start

Generate a LiveAPI client service in Python for my project using Gemini Enterprise, including a test UI for streaming audio and video.

Frequently Asked Questions about liveapi-service

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a client for the Gemini Live API websocket?▼

Provide your project ID, location, model ID, and preferred language, and the Skill generates a service class that opens the websocket connection, sends the setup message, and exposes async send_realtime_data, send_client_content, and receive functions.

What is the difference between Gemini Enterprise and Google AI Live API endpoints?▼

Gemini Enterprise uses a Vertex AI websocket endpoint with bearer token authentication and mediaChunks for realtime input, while Google AI uses the generativelanguage endpoint with an API key and typed audio/video fields. The generated client handles both modes.

How does Live API session resumption work after a disconnect?▼

The client stores the session handle from sessionResumptionUpdate messages and reconnects with it after a goAway signal or websocket error. Sent messages are buffered and replayed on the new connection, with the user message index restarting at 1.

Can I stream audio and video together to the Gemini Live API?▼

Yes, realtime input supports interleaved audio chunks in 16 kHz PCM and video frames as JPEG, PNG, or WebP images. On Gemini Enterprise both are combined in mediaChunks, while Google AI uses separate audio and video fields.

How should interruptions be handled in Live API audio playback?▼

When serverContent reports interrupted, the client must immediately stop audio and transcription playback, clear the playback buffer, and start new conversation bubbles. Audio and transcription must remain time-aligned during normal playback.

Related Skills

Explore More Software Engineering Skills→
2nth-ai2nth-ai
1

tech/google/cloud/ai

Deploy and manage AI/ML models on Google Cloud Vertex AI endpoints.

Official
Advanced
involvexinvolvex
2

firebase-ai-logic

Call Gemini models directly from web and mobile client code.

Community
Intermediate
googlegoogle
19.1k

gemini-live-api

Generates a Gemini Live API WebSocket client with session resumption and demo UI.

Official
Advanced
brivarobrivaro

vertex-ai-api-dev

Guide Gemini API usage on Google Cloud Vertex AI with Gen AI SDKs.

Community
Intermediate