What problem does it solve? Grouped-GEMM MoE stage-2 (down-proj plus top-k reduce) at prefill on AMD gfx950 GPUs is bottlenecked by HBM traffic on the per-token, per-slot partial tensor, which is written once by the GEMM and read once by the reduce kernel at full bf16 width. ## Core Features & Use Cases - fp8 partial store/load: Keeps MFMA compute and accumulation at bf16/f32 while storing only the global partial tensor as fp8 e4m3, halving both the write and read traffic. - Symmetric scale mechanism: Applies a calibrated scale s before the fp8 store and 1/s after the fp8-to-f32 unpack in the reducer, so the scale cancels exactly in the f32 sum. - Validation guidance: Defines runtime signature gates (cshuffle_pf8 store, infp8 reducer), relaxed full-logits parity checks, and segment-level A/B scoring of GEMM plus reduce kernels summed. - Use Case: When profiling a FlyDSL grouped-MoE prefill workload on MI355X shows the stage-2 down/reduce segment is bandwidth-bound, apply this recipe to achieve a measured 1.21x segment speedup with the reduce kernel alone improving 2.05x. ## Quick Start Ask the agent to apply the fp8-partial store and load recipe to the FlyDSL grouped-MoE stage-2 down-proj and reduce kernels, then validate parity and time both kernels together.
