skills.rest
dawsonblock
dawsonblockCOMMUNITY
Explore all skills by this author →
GitHub

serving-llms-vllm

Serve OpenAI-compatible LLM endpoints with vLLM and PagedAttention.

Updated Apr 26, 2026
Search Tags:#quantization#inference#openai-compatible#vllm#llm-serving#tensor-parallelism#paged-attention
One-click install
npx skills add https://github.com/dawsonblock/HERMY --skill serving-llms-vllm-dawsonblock
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: serving-llms-vllm
Source: https://github.com/dawsonblock/HERMY/tree/main/hermes-agent-2026.4.23/skills/mlops/inference/vllm
Command: npx skills add https://github.com/dawsonblock/HERMY --skill serving-llms-vllm-dawsonblock

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Enable production-grade, high-throughput LLM serving with OpenAI-compatible endpoints while minimizing memory usage via vLLM's PagedAttention and continuous batching.

Core Features & Use Cases

  • High-throughput LLM serving with PagedAttention and continuous batching.
  • OpenAI-compatible API endpoints for chat and completions.
  • Support for quantization methods (AWQ, GPTQ, FP8) and tensor parallelism.
  • Production-grade deployment with monitoring, metrics, and multi-model support.
  • Use cases include deploying production APIs, performance benchmarking, and cost-efficient hosting.

Quick Start

Install the required dependencies and run the server with your model to expose a production-ready OpenAI-compatible endpoint.

Frequently Asked Questions about serving-llms-vllm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy an OpenAI-compatible LLM API for production?▼

Deploy an OpenAI-compatible LLM API using vLLM to achieve high-throughput serving. It leverages PagedAttention and continuous batching to minimize memory usage while exposing standard chat and completion endpoints for production workloads.

What's the best way to serve quantized LLMs across multiple GPUs?▼

Serve quantized LLMs across multiple GPUs using vLLM with tensor parallelism. It supports AWQ, GPTQ, and FP8 quantization methods to maximize multi-GPU inference efficiency and reduce overall memory consumption.

How does PagedAttention improve LLM serving throughput?▼

PagedAttention improves LLM serving throughput by optimizing memory allocation during inference. Combined with continuous batching in vLLM, it reduces memory fragmentation and enables significantly higher concurrent request processing.

Can I use vLLM for performance benchmarking and multi-model hosting?▼

You can use vLLM for performance benchmarking and multi-model hosting. It provides production-grade deployment capabilities with built-in monitoring and metrics to evaluate and manage multiple served models efficiently.

Do I need PyTorch and Transformers to run vLLM for inference?▼

You need PyTorch and Transformers to run vLLM for inference. These dependencies are required alongside the vLLM package to load models and execute high-throughput tensor operations during API serving.

Related Skills

Explore More Software Engineering Skills→
rickyanandarickyananda
1

serving-llms-vllm

Serve LLMs with vLLM using OpenAI-compatible APIs and quantization.

Community
Advanced
qcmuuqcmuu

serving-llms-vllm

Deploy vLLM inference endpoints with quantization and tensor parallelism.

Community
Advanced
BagelHoleBagelHole
46

llm-gateway

Deploy a unified API gateway for LLM traffic with load balancing and rate limiting.

Community
Advanced
hung-phanhung-phan
1

vllm

Optimize LLM inference with PagedAttention, continuous batching, and quantization.

Community
Advanced