Install, configure, and run vLLM on NVIDIA GPUs with CUDA.
npx skills add https://github.com/yangwhale/gpu-tpu-pedia --skill vllm-installer
Please help me install this Agent Skill. Skill: vllm-installer Source: https://github.com/yangwhale/gpu-tpu-pedia/tree/main/VibeCoding/claude-code/skills/vllm-installer Command: npx skills add https://github.com/yangwhale/gpu-tpu-pedia --skill vllm-installer
This skill provides a turnkey guide to install, configure, and diagnose vLLM on NVIDIA GPUs, removing guesswork and reducing setup time.
source ./scripts/setup_env.sh pip install vllm==0.14.1 flashinfer-python==0.5.3 flashinfer-cubin==0.5.3 pip install nvidia-nccl-cu12==2.28.3 nvidia-cudnn-cu12==9.16.0.29 vllm serve Qwen/Qwen2.5-7B-Instruct --tensor-parallel-size 4 --port 8000 --host 0.0.0.0
High-intent search queries and answers about installing and using this skill.
To install vLLM on NVIDIA GPUs, run the environment setup script and use pip to install vLLM, FlashInfer, and NCCL packages, ensuring prerequisites like CUDA and PyTorch are correctly configured before launching the server.
Yes, you can run multi-GPU tensor parallelism with vLLM by passing the tensor parallel size argument during server startup, distributing model inference across multiple NVIDIA GPUs.
Yes, vLLM supports MoE models using DeepEP, which is configured during environment setup to ensure readiness and validated through diagnostic workflows before server startup.
If your vLLM server fails to start, run the diagnostic workflow to check LSSD mount configurations, verify LD_LIBRARY_PATH setup, and ensure all CUDA and PyTorch dependencies are correctly installed.
Yes, you need FlashInfer and KV transfer components installed alongside vLLM to properly configure the environment and serve an OpenAI-compatible API endpoint.
The best way to verify a vLLM deployment is to launch an OpenAI-compatible API server and test model endpoints directly, ensuring the environment validation and debugging steps are completed beforehand.
amdCollect OS, Python, GPU, and accelerator facts for Quark installation readiness.
peteromalletDeploy LLM inference with vLLM using OpenAI-compatible endpoints, quantization, and tensor parallelism.
openJiuwen-aiMigrate DeepEP MoE dispatch and combine implementations to Ascend CAM operators.
RheasilviaDeploy OpenAI-compatible LLM inference servers using vLLM with PagedAttention and continuous batching.
