skills.rest
handsomelong922
handsomelong922COMMUNITY
Explore all skills by this author →
GitHub

serving-llms-vllm

Deploy scalable LLM serving with vLLM for OpenAI-compatible endpoints.

Updated Apr 3, 2026
Search Tags:#performance#quantization#openai-api#vllm#llm-serving#tensor-parallelism#production-api
One-click install
npx skills add https://github.com/handsomelong922/my-codex-skills --skill serving-llms-vllm-handsomelong922
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: serving-llms-vllm
Source: https://github.com/handsomelong922/my-codex-skills/tree/main/skills/vllm
Command: npx skills add https://github.com/handsomelong922/my-codex-skills --skill serving-llms-vllm-handsomelong922

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

It enables production-grade, high-throughput LLM serving using vLLM to meet latency and scale requirements for OpenAI-compatible endpoints.

Core Features & Use Cases

  • High-throughput serving: Leverages vLLM features like PagedAttention and continuous batching to maximize throughput.
  • OpenAI-compatible endpoints: Simplifies integration with standard OpenAI API clients and SDKs.
  • Quantization support: Works with GPTQ/AWQ/FP8 to fit large models into constrained GPU memory.
  • Use Case: Deploy a scalable LLM API that handles hundreds of QPS with low latency and robust reliability.

Quick Start

Start a vLLM server with your chosen model to begin serving OpenAI-compatible requests immediately.

Frequently Asked Questions about serving-llms-vllm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy an OpenAI-compatible LLM serving API with vLLM?▼

Deploy an OpenAI-compatible LLM serving API with vLLM by starting a vLLM server with your chosen model. This leverages PagedAttention and continuous batching to immediately handle standard OpenAI API client requests with high throughput.

What is the best way to achieve high throughput for production LLM serving?▼

High throughput for production LLM serving is achieved using vLLM features like PagedAttention and continuous batching. This approach maximizes throughput and minimizes latency, enabling your API to handle hundreds of queries per second robustly.

Can I use quantization to fit large LLMs into constrained GPU memory for serving?▼

You can use quantization to fit large LLMs into constrained GPU memory for serving by applying GPTQ, AWQ, or FP8 formats. vLLM supports these quantization methods to help deploy large models efficiently across available GPUs.

Does vLLM support tensor parallelism for multi-GPU LLM deployments?▼

vLLM supports tensor parallelism for multi-GPU LLM deployments, allowing you to scale large models across multiple GPUs. Configuring tensor parallelism is a required step to maximize throughput and manage memory for production APIs.

How do I monitor and troubleshoot latency issues in a vLLM production API?▼

Monitor and troubleshoot latency issues in a vLLM production API using vLLM monitoring tooling and benchmarking deployment patterns. This helps identify bottlenecks and ensures your high-throughput serving maintains low latency.

What are the limitations of using vLLM for multi-model serving?▼

Limitations of using vLLM for multi-model serving involve managing complex tensor parallelism configurations and quantization formats across different models. Proper vLLM configuration knowledge is required to avoid performance degradation in production APIs.

Related Skills

Explore More Software Engineering Skills→
dawsonblockdawsonblock

serving-llms-vllm

Serve OpenAI-compatible LLM endpoints with vLLM and PagedAttention.

Community
Advanced
mouayadakelmouayadakel

load-test-generator

Generate k6 load testing scripts for API endpoints with test stages and thresholds.

Community
Intermediate
AlvaroBianoAlvaroBiano

serving-llms-vllm

Serve LLMs via vLLM with OpenAI-compatible endpoints.

Community
Advanced
ihatesea69ihatesea69
3

serving-llms-vllm

Serve LLMs with vLLM PagedAttention and continuous batching.

Community
Advanced