sglang
CommunityRadix-attention for ultra-fast LLM serving.
Authorovachiever
Version1.0.0
Installs0
System Documentation
What problem does it solve?
SGLang is a high-performance serving framework for LLMs with RadixAttention prefix caching. It accelerates structured outputs (JSON/regex) and agent workflows by caching shared prefixes across requests.
Core Features & Use Cases
- RadixAttention: automatic KV-prefix caching to accelerate multi-turn prompts
- Structured outputs: JSON, regex-constrained generation
- Agent workflows with function calling and tool integration
- Fast decoding and reduced latency for large prompts
- Production-grade deployment for multi-tenant LLM serving
Use cases include agent-driven tasks, chatbots with repeated prefixes, and high-throughput inference pipelines.
Quick Start
Run a local server and expose a JSON-structured output endpoint, then integrate with a client to perform batch generations.
Dependency Matrix
Required Modules
sglangtorchtransformers
Components
references
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: sglang Download link: https://github.com/ovachiever/droid-tings/archive/main.zip#sglang Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.