skills.rest
togethercomputer
togethercomputerOFFICIAL
Explore all skills by this author →
GitHub

diffusion-optimal-perf

Tune CLI flags and offload strategies to reduce latency and VRAM usage in SGLang-Diffusion.

Updated Mar 17, 2026
Search Tags:#performance#benchmarks#inference#diffusion#sglang#torch-compile#gpu-optimizations
One-click install
npx skills add https://github.com/togethercomputer/xorl-sglang --skill diffusion-optimal-perf-togethercomputer
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: diffusion-optimal-perf
Source: https://github.com/togethercomputer/xorl-sglang/tree/main/python/sglang/multimodal_gen/.claude/skills/diffusion-optimal-perf
Command: npx skills add https://github.com/togethercomputer/xorl-sglang --skill diffusion-optimal-perf-togethercomputer

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Diffusion inference is often latency-bound and memory-intensive, hindering rapid experimentation and production deployment.

Core Features & Use Cases

  • Provides a structured set of lossless and lossy optimizations, including Torch Compile, warmup strategies, multi-GPU parallelism, and selective offloads.
  • Supports dynamic and static configuration for single- and multi-GPU setups, enabling lower latency and reduced VRAM usage across production workloads.
  • Use Case: You have a diffusion model running on 8 GPUs and must minimize inference latency while keeping VRAM under a strict limit.

Quick Start

Run sglang generate with a diffusion model path and the recommended optimization flags to start benchmarking performance improvements.

Frequently Asked Questions about diffusion-optimal-perf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize diffusion inference latency using SGLang?▼

Optimize SGLang diffusion inference by tuning CLI flags, environment variables, and offload strategies. Applying lossless and lossy optimizations like Torch Compile and warmup strategies reduces latency and VRAM usage.

What's the best way to reduce VRAM usage for diffusion models on multiple GPUs?▼

Reduce VRAM usage on multiple GPUs by configuring multi-GPU parallelism and selective offloads in SGLang. Dynamic and static configurations enable scalable deployment while keeping memory under strict limits.

Does SGLang diffusion optimization support Torch Compile?▼

Yes, SGLang diffusion optimization supports Torch Compile. A compatible PyTorch environment with Torch Compile support is required to apply these lossless and lossy performance optimizations.

How does Torch Compile improve diffusion model performance?▼

Torch Compile improves diffusion model performance through lossless graph optimizations. Combined with warmup strategies, it accelerates inference and minimizes latency bottlenecks during rapid experimentation and production deployment.

Can I benchmark diffusion performance improvements with SGLang?▼

Yes, you can benchmark diffusion performance improvements with SGLang. Run sglang generate with a diffusion model path and recommended optimization flags to measure inference latency and VRAM reductions.

What are the limitations of lossy optimizations for SGLang diffusion?▼

Lossy optimizations for SGLang diffusion may trade output fidelity for lower latency and reduced VRAM. Evaluate benchmark results carefully to ensure model quality remains acceptable for your production workloads.

Related Skills

Explore More Software Engineering Skills→
annealing-inversionannealing-inversion

sglang-diffusion-modelopt-quant

Quantize diffusion DiT models with NVIDIA ModelOpt for SGLang Diffusion.

Community
Advanced
moirai-internalmoirai-internal

diffusion-optimal-perf

Optimize SGLang-Diffusion inference with CLI flags and environment variables.

Official
Advanced
spacecat2002spacecat2002

sglang-diffusion-performance

Select SGLang Diffusion configuration flags for generation speed and GPU memory usage.

Community
Advanced
BermudaLocalsBermudaLocals
1

tensorrt-llm

Optimize LLM inference with TensorRT-LLM on NVIDIA GPUs using quantization and multi-GPU scaling.

Community
Advanced