generate-profile

To profile an SGLang server for GPU inference latency, you need to generate an end-to-end performance tracing trace. This process launches the server, runs readiness checks, executes a GSM8K sanity test, and captures the profile output.

Capturing prefill and decode performance traces separately for SGLang involves configuring specific profiling steps during server validation. The trace generation isolates these stages to help investigate latency and throughput bottlenecks in GPU-backed deployments.

Yes, you need a working SGLang installation and an available CUDA GPU to run performance benchmarking and trace generation. These are required to validate server health, execute the GSM8K sanity test, and produce Chrome-compatible trace output.

Yes, SGLang profiling output is Chrome-compatible and works directly with Perfetto or Chrome tracing. The generated trace file allows you to inspect prefill and decode stages to identify serving bottlenecks in your GPU deployment.

SGLang server profiling requires a GSM8K accuracy sanity test to validate that the deployment is functioning correctly before capturing traces. This ensures the performance trace reflects accurate model serving behavior rather than startup errors.

Yes, you can configure tensor parallelism, GPUs, ports, and custom output paths when profiling SGLang runs. Server orchestration supports flexible analysis by allowing you to specify model configurations and direct the trace output to investigate specific performance issues.