Configure and run vLLM for high-throughput LLM inference on GPUs.
npx skills add https://github.com/tylertitsworth/skills --skill vllm
Please help me install this Agent Skill. Skill: vllm Source: https://github.com/tylertitsworth/skills/tree/main/vllm Command: npx skills add https://github.com/tylertitsworth/skills --skill vllm
vLLM enables high-throughput, memory-efficient LLM inference on GPUs, enabling scalable deployment of large models with advanced memory management techniques such as PagedAttention.
Start the server with vllm serve using your model ID to begin HTTP API access.
High-intent search queries and answers about installing and using this skill.
High-throughput LLM inference is configured using vLLM's Python kwargs or CLI to enable tensor and pipeline parallelism across multiple GPUs. This manages memory efficiently and scales large model serving in cloud-native environments.
PagedAttention improves KV cache management by optimizing memory allocation during GPU inference, reducing memory fragmentation. This enables higher throughput and allows larger batch sizes when serving large language models.
Yes, you can serve chat completions and embeddings via an OpenAI-compatible API. The server supports serving multiple endpoints while optionally allowing LoRA adapters, speculative decoding, and structured outputs.
Kubernetes vLLM deployment supports disaggregated prefill and decode serving, along with LMCache offloading. Guidance for these advanced configurations is provided within the included references.
The best way to start serving is by executing the vllm serve command with your target model ID. This initiates HTTP API access for high-throughput inference on configured GPU hardware.
nebiusProvision Nebius cloud resources and manage IAM authentication for physical-AI workflows.
microwindDesign API gateways for routing, security, and traffic management in microservices.
Supporter09Accelerate LLM token generation with speculative decoding strategies.
aws-samplesAutomate vLLM inference serving deployment on EKS Kubernetes clusters.
