skills.rest
saintgo7
saintgo7COMMUNITY
Explore all skills by this author →
GitHub

gem-llm-deploy-vllm

Manages lifecycle of GEM-LLM vLLM servers including health checks via /v1/models endpoints.

Updated Apr 4, 2026
Search Tags:#deployment automation#health check#gpu memory#vllm#model serving#port conflict#openai-compatible endpoints
One-click install
npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-deploy-vllm
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gem-llm-deploy-vllm
Source: https://github.com/saintgo7/claude-skills/tree/main/gem-llm-deploy-vllm
Command: npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-deploy-vllm

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill resolves operational friction when starting, stopping, and verifying GEM-LLM vLLM model servers, including common issues like port conflicts and GPU memory (OOM) failures.

Core Features & Use Cases

  • Two-model vLLM lifecycle control: Starts, stops, and performs health checks for the main Gemma-based server and an auxiliary model server.
  • Single-node GPU-aware configuration: Generates vLLM launch settings that respect the single-node constraint (tensor-parallel sizing and safe launch parameters) for the 8xB200 environment.
  • Operations-oriented troubleshooting: Helps diagnose endpoint health via /v1/models, inspects GPU status with nvidia-smi, and guides responses to port collisions and CUDA OOM scenarios using the expected log location.

Quick Start

Run the skill installer with gem-llm-deploy-vllm to start or restart the vLLM main and auxiliary servers, then verify readiness by checking both /v1/models endpoints.

Frequently Asked Questions about gem-llm-deploy-vllm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I restart a vLLM server and verify its OpenAI-compatible endpoints?▼

To restart vLLM servers, trigger the lifecycle management command to start or stop the main and auxiliary model servers, then validate readiness by checking the OpenAI-compatible `/v1/models` responses for both endpoints.

Why does vLLM fail to start with a GPU OOM error on a multi-GPU node?▼

vLLM fails with GPU OOM when `gpu-memory-utilization` is misconfigured or tensor-parallel sizing exceeds available VRAM. You must correctly configure tensor-parallel parameters and dtype settings to respect the single-node 8xB200 GPU memory constraints during deployment.

How can I troubleshoot port conflicts when launching multiple vLLM model servers?▼

To troubleshoot vLLM port conflicts, inspect the server logs located at `/home/jovyan/gem-llm/_logs/vllm-*.log` to identify binding errors. The lifecycle manager helps diagnose and resolve port collisions between the main and auxiliary model serving endpoints.

Do I need specific tensor-parallel settings for vLLM on an 8xB200 single-node environment?▼

Yes, you need specific tensor-parallel sizing for vLLM on an 8xB200 single-node environment. The deployment requires correct tensor-parallel configurations and safe `gpu-memory-utilization` parameters to ensure both main and auxiliary servers launch without exceeding GPU memory limits.

Where are vLLM server logs stored for debugging deployment issues?▼

vLLM server logs are stored in the `/home/jovyan/gem-llm/_logs/vllm-*.log` directory. Checking these log files is the primary method for diagnosing startup failures, port conflicts, and CUDA OOM errors during vLLM model serving operations.

Related Skills

Explore More Software Engineering Skills→
saintgo7saintgo7

gem-llm-vllm-debug

Diagnose vLLM startup and runtime failures with validated dependency and launch configurations.

Community
Advanced
rblake2320rblake2320

nemoclaw-user-configure-inference

Configure NemoClaw to route inference to local Ollama, vLLM, or NIM servers.

Community
Advanced
lawless-mlawless-m

Vram-GPU-OOM

Coordinate GPU VRAM usage across Ollama, Whisper, and ComfyUI with OOM retry and auto-unload.

Community
Advanced
Strategic-AutomationStrategic-Automation
6

serving-llms-vllm

Orchestrate vLLM-based inference for scalable, low-latency LLM serving.

Official
Advanced