serving-llms-vllm

Deploy an OpenAI-compatible LLM inference server using vLLM by configuring high-throughput endpoints for chat and completions. This Skill provides serving commands, performance settings, and verification steps to launch production-ready APIs for large language models.

Yes, you can fit larger models into smaller GPU budgets using vLLM quantization. This Skill supports AWQ, GPTQ, and FP8 quantization techniques to reduce memory constraints and deploy large language models efficiently.

Improve vLLM throughput and latency for production APIs by applying continuous batching, prefix caching, tensor parallelism, and speculative decoding. This Skill configures these performance optimizations for low-latency chat endpoints and batch generation workflows.

Yes, vLLM supports multi-GPU and multi-node serving scenarios using tensor parallelism. This Skill guides GPU-backed deployments, configuring distributed execution to scale high-throughput large language model inference across multiple GPUs.

Set up monitoring and troubleshooting for a vLLM deployment by configuring Docker, Kubernetes, health checks, and failure recovery procedures. This Skill provides operational guidance to maintain reliable large language model serving environments.