Configure SGLang HiCache tiered KV caching with per-rank sizing and eviction policies.
npx skills add https://github.com/air-gapped/skills --skill sglang-hicache
Please help me install this Agent Skill. Skill: sglang-hicache Source: https://github.com/air-gapped/skills/tree/main/.claude/skills/sglang-hicache Command: npx skills add https://github.com/air-gapped/skills --skill sglang-hicache
SGLang HiCache enables hierarchical KV caching to extend per-rank GPU memory with L2 host RAM and optional L3 storage, unlocking larger models and longer context.
Start a SGLang server with hierarchical cache enabled and pick a backend (e.g., Mooncake) using per-rank sizing and a production-safe prefetch policy.
High-intent search queries and answers about installing and using this skill.
SGLang HiCache extends per-rank KV storage by implementing a three-tier KV cache (L1/L2/L3), utilizing L2 host RAM and optional L3 storage to support larger models and longer contexts.
You can use multiple L3 backends for distributed KV cache storage including mooncake, hf3fs, nixl, aibrix, eic, simm, and file, with support for runtime attach and detach to swap backends dynamically.
Yes, SGLang HiCache supports per-rank sizing, configurable eviction policies, and prefetch controls to enforce tiered caching behavior for multi-tenant inference across distributed backends.
Start a SGLang server with hierarchical cache enabled by selecting an L3 backend such as Mooncake, then configuring per-rank sizing and a production-safe prefetch policy using the provided recipes.
Yes, Python3 and curl are required dependencies to configure SGLang HiCache for hybrid-model deployments, utilizing provided scripts and references for production deployment recipes.
hoangnh2412Configure Redis distributed caching for Jarvis ASP.NET Core applications.
marquesfelipMonitor per-tenant SaaS metrics with Prometheus dashboards and PostgreSQL aggregates.
air-gappedCoordinate vLLM KV caching across native offload, LMCache, Nixl, and Mooncake backends.
cyberpunk042Configure per-model prompt and KV cache settings in LocalAI YAML files.
