gluon-pipeline-opt

You can hide global memory latency in Gluon GEMM kernels by applying a two-stage pipeline optimization with double buffering, overlapping data loads with compute to maintain high MFMA throughput on CDNA3 and CDNA4 architectures.

The best way to overlap ds_read latency with MFMA on MI300X is to implement Stage 2 local prefetch in your Gluon GEMM kernel, an architecture-agnostic optimization that reduces lgkmcnt stalls and improves compute efficiency.

Yes, the dual-stage prefetch optimization supports both CDNA3 and CDNA4 architectures by using architecture-specific paths, utilizing async_copy for CDNA4 and buffer_load for CDNA3 to enable global prefetch and double buffering.

To apply two-stage pipeline optimizations, first implement Stage 1 for global prefetch and double buffering, verify correctness with included checks, then apply Stage 2 to overlap ds_read with MFMA and validate performance using traces.

Your Gluon GEMM kernel experiences vmcnt stalls on gfx942 due to exposed global memory latency; applying Stage 1 pipeline optimizations with double buffering hides this latency to improve MFMA utilization.