skills.rest
tadod12
tadod12COMMUNITY
Explore all skills by this author →
GitHub

serving-llms-vllm

Serve LLM inference via OpenAI-compatible endpoints using vLLM.

Updated Mar 18, 2026
Search Tags:#production#llm#throughput#quantization#inference#vllm#serving
One-click install
npx skills add https://github.com/tadod12/fraud-detection-research --skill serving-llms-vllm-tadod12
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: serving-llms-vllm
Source: https://github.com/tadod12/fraud-detection-research/tree/main/.agent/skills/12-inference-serving/vllm
Command: npx skills add https://github.com/tadod12/fraud-detection-research --skill serving-llms-vllm-tadod12

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Scales production LLM inference by serving models via vLLM with high throughput, low latency, and memory-efficient batching, enabling production-grade AI endpoints.

Core Features & Use Cases

  • OpenAI-compatible API endpoints for seamless integration with existing applications.
  • High-throughput inference using vLLM's PagedAttention and continuous batching.
  • Support for model quantization (AWQ/GPTQ/FP8) and tensor parallelism to fit large models on constrained GPUs.
  • Production deployment patterns, reference guides, and troubleshooting resources to operate AI services reliably.

Quick Start

Launch a production-ready vLLM server that exposes an OpenAI-compatible API and supports quantization and multi-GPU setups.

Frequently Asked Questions about serving-llms-vllm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy an OpenAI-compatible LLM inference server with vLLM?▼

Deploy an OpenAI-compatible LLM inference server with vLLM by launching a production-ready server that exposes standard API endpoints, enabling seamless integration with existing applications requiring high-throughput text generation.

Can I serve quantized LLM models using vLLM to fit constrained GPUs?▼

Serve quantized LLM models using vLLM to fit constrained GPUs by applying AWQ, GPTQ, or FP8 quantization formats, reducing memory consumption while maintaining production-grade inference throughput.

How does vLLM achieve high throughput for production LLM serving?▼

vLLM achieves high throughput for production LLM serving by utilizing PagedAttention and continuous batching mechanisms, enabling memory-efficient batching and low latency across multi-GPU setups.

What is the best way to scale LLM inference across multi-GPU setups?▼

Scale LLM inference across multi-GPU setups by applying tensor parallelism patterns in vLLM, distributing model execution across multiple GPUs to maximize throughput and handle large quantized models.

Does vLLM support Kubernetes and Docker deployment patterns for production?▼

vLLM supports production deployment patterns for Docker and Kubernetes, providing reference guides and troubleshooting resources to operate AI services reliably with load balancing and monitoring configurations.

Why should I use vLLM for production LLM serving instead of standard inference engines?▼

Use vLLM for production LLM serving to achieve high throughput, low latency, and memory-efficient batching via PagedAttention, scaling inference across multi-GPU setups with quantized models better than standard engines.

Related Skills

Explore More Software Engineering Skills→
john-data-chenjohn-data-chen
2

serving-llms-vllm

Configure vLLM servers for OpenAI-compatible endpoints with Docker or Kubernetes.

Community
Advanced
superfhpsuperfhp

serving-llms-vllm

Serve LLMs with OpenAI-compatible endpoints using vLLM.

Community
Advanced
ovachieverovachiever
52

serving-llms-vllm

Deploy OpenAI-compatible LLM APIs with vLLM for high-throughput inference.

Community
Advanced
GregsGreyCodeGregsGreyCode
6

serving-llms-vllm

Deploy vLLM servers with PagedAttention, continuous batching, and OpenAI-compatible endpoints.

Community
Advanced