diffusion-kernel

Optimize GPU kernel performance for diffusion models by integrating highly efficient CUDA and Triton kernels. This Skill targets bottlenecks in elementwise operations, normalization, attention, and memory access patterns to maximize inference speed and throughput.

Speed up inference for diffusion model components by applying optimized CUDA and Triton kernels for operations like RMSNorm, AdaLN, and RoPE. These specialized kernels replace default implementations to reduce latency and improve overall generation throughput.

Profile and benchmark custom CUDA kernels with Nsight Compute using the provided tools. The Skill includes detailed guides for profiling performance against baselines and writing custom kernels optimized for NVIDIA GPUs like H100, A100, and T4.

Writing custom Triton kernels improves diffusion model inference acceleration by addressing specific memory access pattern bottlenecks. Optimizing these elementwise operations and attention mechanisms directly reduces latency and speeds up image or video generation cycles.

NVIDIA H100 or A100 GPUs are not strictly required to see performance gains from diffusion kernel optimization. The provided CUDA and Triton kernels also support the T4, allowing inference acceleration and throughput improvements across various NVIDIA GPU architectures.