lmcache-mp
OfficialLMCache MP for scalable KV caching across pods
Authorair-gapped
Version1.0.0
Installs0
System Documentation
What problem does it solve?
LMCache multiprocess (MP) mode runs the KV-cache server in its own process so multiple vLLM pods on a node can share a centralized cache with independent CPU/memory budgets and a ZMQ-based transport, enabling scalable inference at scale.
Core Features & Use Cases
- Separate LMCache server per node that serves multiple vLLM pods on the same node, reducing GPU contention and improving cache hit rates for multi-pod deployments.
- Kubernetes-friendly deployment pattern: a DaemonSet for LMCache and a Deployment for vLLM pods, using L1/L2 storage cascades (nixl_store, nixl_store_dynamic, fs, mooncake_store, s3) as configured.
- Production-ready references: operator-based reconciliation and example manifests, plus detailed deployment and troubleshooting guidance.
Quick Start
Start a LMCache MP DaemonSet on your cluster and configure vLLM to connect to the LMCache MP port.
Dependency Matrix
Required Modules
None requiredComponents
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: lmcache-mp Download link: https://github.com/air-gapped/skills/archive/main.zip#lmcache-mp Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 510,000+ vetted skills library on demand.