Serve high-throughput LLM inference with OpenAI-compatible APIs via vLLM.
npx skills add https://github.com/kevinnft/ai-agent-skills --skill vllm-kevinnft
Please help me install this Agent Skill. Skill: vllm Source: https://github.com/kevinnft/ai-agent-skills/tree/main/skills/mlops/inference/vllm Command: npx skills add https://github.com/kevinnft/ai-agent-skills --skill vllm-kevinnft
vLLM enables high-throughput, OpenAI-compatible LLM serving for production environments, reducing latency and enabling scalable deployments.
Install vLLM and start a server with your model using a single command.
High-intent search queries and answers about installing and using this skill.
Serve LLMs with an OpenAI-compatible API in production using vLLM to achieve high-throughput inference. It applies paged KV cache and continuous batching to reduce latency for scalable endpoints.
vLLM supports multi-GPU tensor parallelism and scalable Kubernetes deployments. It provides production-ready deployment patterns using Docker and multi-node setups with monitoring capabilities.
Fit large LLMs into limited VRAM during inference by applying quantization. vLLM supports AWQ, GPTQ, and FP8 quantization options to reduce memory footprint and accelerate serving.
Continuous batching improves high-throughput LLM inference by dynamically processing incoming requests. vLLM pairs this with paged KV cache management to maximize GPU utilization and reduce latency.
vLLM supports troubleshooting and monitoring for production serving. It includes documented deployment patterns and performance optimization guidance for maintaining scalable inference environments.
nadicodeaiLaunch vLLM servers with OpenAI-compatible endpoints and tensor parallelism for LLM inference.
AveerOrchestrate deployment and configuration of local LLM inference servers for OpenCode.
shuff57Serve OpenAI-compatible LLM APIs with vLLM and quantization support.
heysuhasDeploy LLM inference servers with the vLLM engine and OpenAI-compatible APIs.
