Bootstrap an OpenAI-compatible vLLM server with validated dependencies and smoke tests.
npx skills add https://github.com/saintgo7/claude-skills --skill vllm-bootstrap
Please help me install this Agent Skill. Skill: vllm-bootstrap Source: https://github.com/saintgo7/claude-skills/tree/main/vllm-bootstrap Command: npx skills add https://github.com/saintgo7/claude-skills --skill vllm-bootstrap
vLLM servers often fail to start or behave incorrectly due to dependency/version mismatches, wrong tensor-parallel settings, tool-calling parser incompatibilities, and GPU memory/OOM issues during model loading and request handling.
Ask an AI to guide you to install the verified dependencies and launch an OpenAI-compatible vLLM server with a chosen tensor-parallel size and tool-call parser that matches your model, then run the /v1/models and chat/tool-calling smoke tests to confirm it is working.
High-intent search queries and answers about installing and using this skill.
Fix vLLM startup failures by installing a verified dependency matrix that specifies compatible versions for vLLM, transformers, flashinfer, and mistral_common to avoid known-bad combinations.
Resolve GPU OOM during vLLM model loading or serving by tuning GPU memory utilization and adjusting tensor-parallel size values like TP=1, TP=2, TP=4, or TP=8 to distribute memory load.
Tool calls are not parsed when the parser mismatches the model; map the correct tool-call parser such as hermes, mistral, or pythonic to your specific model family to resolve parsing failures.
Verify a vLLM deployment is working by running curl-based smoke tests against the /v1/models endpoint and /v1/chat/completions, including tool-calling tests, plus logging and GPU metric checks.
Using TP=8 with vLLM can cause NCCL hangs if configuration is incorrect; apply the failure-to-fix playbook to remediate NCCL TP hangs, flashinfer cubin mismatches, and port conflicts.
vLLM supports OpenAI-compatible API endpoints for production serving, allowing you to boot a reliable server and test chat completions and tool calling through standard API requests.
saintgo7Diagnose vLLM startup and runtime failures with validated dependency and launch configurations.
saintgo7Run inference and tool-calling smoke tests against a local GEM-LLM vLLM endpoint.
Strategic-AutomationOrchestrate vLLM-based inference for scalable, low-latency LLM serving.
davpatel605-beepDeploy and optimize vLLM inference servers with OpenAI-compatible APIs.
