cae-streaming

Streams Kit-CAE over WebRTC and handles remote data-channel control messages.

62|22|Updated Aug 19, 2025
One-click install
npx skills add https://github.com/NVIDIA-Omniverse/kit-cae --skill cae-streaming-nvidia-omniverse
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: cae-streaming
Source: https://github.com/NVIDIA-Omniverse/kit-cae/tree/main/skills/cae-streaming
Command: npx skills add https://github.com/NVIDIA-Omniverse/kit-cae --skill cae-streaming-nvidia-omniverse

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, and includes scripts (resource) and references (resource) components.

What problem does it solve? Running Kit-CAE as an interactive application that remote clients can watch and control requires wiring WebRTC streaming, a JSON wire protocol, and long-lived message handlers by hand. This Skill packages the streaming-enabled .kit configuration, the protocol references, and a reusable Python handler framework so a browser, agent, or dashboard can view and drive a live Kit-CAE session. ## Core Features & Use Cases - WebRTC streaming server: Launch a headless, viewport-only Kit-CAE app via scripts/launch_streaming.sh with the omni.kit.livestream.messaging extension preconfigured. - Request/response handler framework: scripts/omniverse_api.py provides @api.request and @api.signal decorators that handle sync and async handlers, payload unpacking, id correlation, and error-to-JSON conversion. - Runnable reference handler: scripts/serve.py implements load_scene, list_scenes, register_scene, and deregister_scene against a scene registry, demonstrating the end-to-end pattern. - Use Case: An ML pipeline needs to load CAE datasets into a live Omniverse viewport and push results to a browser dashboard; it connects over the WebRTC data channel, sends load_scene_request messages, and receives responses plus scene_loaded signals. ## Quick Start Run bash skills/cae-streaming/scripts/launch_streaming.sh from the Kit-CAE repo root, then connect a WebRTC client and send a load_scene_request message for the static_mixer scene.

Frequently Asked Questions about cae-streaming

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I stream Kit-CAE to a browser over WebRTC?

Run bash skills/cae-streaming/scripts/launch_streaming.sh from the Kit-CAE repo root to launch a streaming app with serve.py as the listener. Connect a browser client using @nvidia/omniverse-webrtc-streaming-library against signaling port 49100.

How do I write a remote control handler for Omniverse Kit?

Decorate a sync or async function with @api.request from scripts/omniverse_api.py; the framework registers <name>_request and <name>_response events, unpacks the payload, awaits coroutines, and converts exceptions into {"ok": false, "error": ...} responses.

Why does my async handler return 'Object of type coroutine is not JSON serializable'?

The observe_and_dispatch decorator in omni.kit.livestream.messaging captures the immediate return value and does not await coroutines. Use the OmniverseAPI @api.request decorator, which schedules async handlers via run_coroutine, or split into a sync receiver plus manual dispatch_event.

Can multiple clients control the same Kit-CAE streaming session?

Yes, but LivestreamMessaging is per-app, not per-session, so all connected clients see and control the same scene. There is no per-client isolation; each load_scene call replaces the previously loaded prims on the shared stage.

When should I use cae-capture instead of WebRTC streaming?

Use cae-capture for headless offline rendering of clean PNG, EXR, or MP4 outputs. Streaming is the live interactive path for clients that need a real-time viewport plus programmatic control, not batch image generation.