skills.rest
saintgo7
saintgo7COMMUNITY
Explore all skills by this author →
GitHub

gem-llm-vllm-debug

Diagnose vLLM startup and runtime failures with validated dependency and launch configurations.

Updated Apr 4, 2026
Search Tags:#debugging#tool calling#vllm#flashinfer#gpu oom#deepgemm#openai compatible api server
One-click install
npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-vllm-debug
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gem-llm-vllm-debug
Source: https://github.com/saintgo7/claude-skills/tree/main/gem-llm-vllm-debug
Command: npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-vllm-debug

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you quickly identify and resolve vLLM model-server startup and runtime failures, including dependency mismatches and GPU-related crashes, so your Claude Code workflow can resume without long downtime.

Core Features & Use Cases

  • Compatibility & dependency matrix: Provides a validated version set (Python, vLLM, transformers, flashinfer, etc.) tailored to common failure patterns.
  • Dual-model vLLM launch templates: Supports operational debugging for Qwen2.5-Coder-32B and Qwen3-Coder-30B-A3B with correct tensor-parallel settings and tool-calling parser configuration.
  • Failure-mode playbook: Covers frequent issues like option incompatibilities, flashinfer version mismatches, DeepGEMM backend unavailability, transformers/mistral_common API breaks, model architecture recognition errors, port conflicts, and leftover GPU memory after crashes.

Quick Start

Run the installer hook for gem-llm-vllm-debug to use the validated debugging setup for your current environment.

Frequently Asked Questions about gem-llm-vllm-debug

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
Why does my vLLM server fail to start with a flashinfer version mismatch?▼

vLLM startup failures from flashinfer mismatches require strict version alignment across vLLM, transformers, and flashinfer, resolved by applying a validated compatibility matrix to your environment.

How do I configure tool calling when serving Qwen2.5-Coder-32B in vLLM?▼

To configure tool calling for Qwen2.5-Coder-32B in vLLM, apply correct launch templates specifying tensor-parallel settings and tool-calling parser configuration for your OpenAI compatible API server.

What is the best way to fix GPU OOM errors and leftover memory remnants after a vLLM crash?▼

Fix GPU OOM errors and leftover memory remnants after a vLLM crash by executing actionable recovery steps that clear residual GPU memory before relaunching the model server.

Does vLLM support DeepGEMM backend unavailability errors during model serving?▼

DeepGEMM backend unavailability errors during vLLM serving indicate dependency misconfigurations, requiring strict version alignment and correct launch parameters to restore backend functionality.

How to resolve port conflicts when launching a vLLM OpenAI compatible API server?▼

Resolve port conflicts when launching a vLLM OpenAI compatible API server by applying correct launch parameters that specify alternative ports and clear processes occupying the default server configuration.

Can I use vLLM to serve Qwen3-Coder-30B-A3B without encountering model architecture recognition errors?▼

Serving Qwen3-Coder-30B-A3B in vLLM without model architecture recognition errors requires strict version alignment of transformers and applying correct dual-model launch templates for configuration.

Related Skills

Explore More Software Engineering Skills→
saintgo7saintgo7

vllm-bootstrap

Bootstrap an OpenAI-compatible vLLM server with validated dependencies and smoke tests.

Community
Advanced
cabeznocabezno

serving-llms-vllm

Deploy an OpenAI-compatible LLM inference server using vLLM.

Community
Advanced
saintgo7saintgo7

gem-llm-deploy-vllm

Manages lifecycle of GEM-LLM vLLM servers including health checks via /v1/models endpoints.

Community
Intermediate
drdave-flexnetosdrdave-flexnetos

AI Inference & Model Serving

Deploy LocalAI and vLLM servers for OpenAI-compatible chat and embeddings.

Community
Intermediate