sglang-runtime-context

Documents SGLang's RuntimeContext architecture for config bags, overrides, and per-forward state.

33.0k|8.4k|Updated Jan 8, 2024
One-click install
npx skills add https://github.com/sgl-project/sglang --skill sglang-runtime-context
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: sglang-runtime-context
Source: https://github.com/sgl-project/sglang/tree/main/.claude/skills/sglang-runtime-context
Command: npx skills add https://github.com/sgl-project/sglang --skill sglang-runtime-context

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

SGLang's process-global runtime state is organized into strict tiers (published ServerArgs seed, namespace config bags, runtime flags, resources, per-forward state), and reading or mutating the wrong tier causes subtle bugs that tests often miss. This Skill explains the architecture, the CI guardrails that enforce it, and the correct idioms so changes to server_args, model overrides, or module-level state don't break the design.

Core Features & Use Cases

  • Config read/write rules: Explains when to read namespace bags via get_exec()/get_schedule()/get_model(), when get_server_args() is a ratchet failure, and how get_context().override() is the only post-publish mutation path.
  • Guardrail catalog: Documents the CI ratchets (read ratchet, exposure ratchet, publish-order tests, role namespace enforcement) so new code passes them on the first try.
  • Testing idioms: Shows how test doubles publish via override_server_args instead of injecting SimpleNamespace config stubs.
  • Use Case: Before adding a new ServerArgs field or moving load-time logic into resolution, load this Skill to pick the correct tier, declaration mechanism, and accessor pattern.

Quick Start

Load the sglang-runtime-context skill before modifying server_args, model overrides, or any module-level runtime state in the sglang repository.

Frequently Asked Questions about sglang-runtime-context

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I read SGLang server_args config values in business code?

Read resolved values from namespace config bags such as get_exec(), get_schedule(), or get_model() rather than get_server_args().field, which is a read-ratchet failure. Bag leaves are plain attributes, safe inside torch.compile-traced code, and reflect post-publish overrides.

How do I mutate SGLang runtime config after publish?

Use get_context().override(source, **fields), the only post-publish mutation entry point. It writes bag leaves in place and records provenance; the ServerArgs instance stays pristine with no write-through.

Why does get_server_args().field fail CI checks in sglang?

A read ratchet pins direct ServerArgs field reads in business code at zero because the record holds raw operator input, not resolved values. Read the corresponding namespace bag leaf or a runtime_context derived accessor instead.

How do I add a model-specific config override in SGLang?

Declare it in sglang/srt/arg_groups/overrides.py: use MODEL_OVERRIDES for constants, @register_model_override for derived values, or a post-process pass for normalization. Never assign server_args fields from model code; only resolvable=True fields are declarable.

How should tests stub SGLang config values?

Tests should publish config via get_context().override_server_args(field=...) and install it, rather than injecting SimpleNamespace stand-ins. Production code reads bags, so injected stubs stop working once a reader migrates to bag accessors.

When should logic stay at load time instead of resolution time in SGLang?

Logic consulting extensible registries, like out-of-tree platforms registering attention backends at model_runner import, must stay at load time in ModelRunner init and write via get_context().override(). __post_init__ runs before any model or platform import.