diffusion-optimal-perf

To optimize SGLang diffusion model inference for lower latency, apply lossless techniques like torch.compile, warmup, and multi-GPU parallelism, or use lossy methods like approximate attention and model quantization to significantly reduce processing delays.

You can reduce diffusion model VRAM usage in SGLang by enabling CPU offloading for specific components and applying model quantization, which trade a small amount of output quality for significant memory footprint reductions.

Lossless diffusion inference speedups use torch.compile, warmup, and multi-GPU parallelism without sacrificing output quality, while lossy speedups utilize approximate attention and quantized models to trade a small amount of quality for significant speed or VRAM gains.

Yes, SGLang supports multi-GPU parallelism for diffusion model inference as a lossless optimization technique, allowing you to distribute workloads across multiple GPUs to accelerate generation without degrading output quality.

You can configure SGLang-Diffusion performance by setting specific CLI flags and environment variables that toggle torch.compile, CPU offloading, approximate attention, and model quantization to achieve your desired latency and VRAM targets.

Use approximate attention when you need maximum speed or VRAM reductions and can tolerate a small quality loss, whereas torch.compile is a lossless optimization better suited for maintaining output fidelity while accelerating diffusion inference.