vllm-omni-multimodal

Unified Omni models enable multimodal AI inference across text, image, audio, and video inputs. This skill orchestrates end-to-end reasoning and generation for interactive chat and content analysis tasks.

Deploying an omni-model for serving multimodal requests is supported through both offline and online options. The skill provides quick-start examples and model configuration guidance for serving.

Multimodal reasoning for video and audio inputs works by passing media directly to the Omni model. The model transcribes speech from video and describes images while answering contextual questions.

Running different omni-model variants requires specific hardware based on VRAM requirements. The skill supports multiple model variants with different VRAM needs and serving modes.

You can use a single Omni model for image description and speech transcription. The skill unifies text, image, audio, and video processing capabilities under one workflow to simplify inference.