gem-llm-deploy-vllm

What problem does it solve?

This Skill resolves operational friction when starting, stopping, and verifying GEM-LLM vLLM model servers, including common issues like port conflicts and GPU memory (OOM) failures.

Core Features & Use Cases

• Two-model vLLM lifecycle control: Starts, stops, and performs health checks for the main Gemma-based server and an auxiliary model server.
• Single-node GPU-aware configuration: Generates vLLM launch settings that respect the single-node constraint (tensor-parallel sizing and safe launch parameters) for the 8xB200 environment.
• Operations-oriented troubleshooting: Helps diagnose endpoint health via /v1/models, inspects GPU status with nvidia-smi, and guides responses to port collisions and CUDA OOM scenarios using the expected log location.

Quick Start

Run the skill installer with gem-llm-deploy-vllm to start or restart the vLLM main and auxiliary servers, then verify readiness by checking both /v1/models endpoints.