flydsl_fp8_blockscale_gemm

Rewrites CK fp8 blockscale GEMM kernels to FlyDSL on gfx950 with software fp32 scaling.

178|52|Updated Jul 30, 2025
One-click install
npx skills add https://github.com/AMD-AGI/GEAK --skill flydsl-fp8-blockscale-gemm-amd-agi
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: flydsl_fp8_blockscale_gemm
Source: https://github.com/AMD-AGI/GEAK/tree/main/perf_knowledge/expert_skills/skills/flydsl_fp8_blockscale_gemm
Command: npx skills add https://github.com/AMD-AGI/GEAK --skill flydsl-fp8-blockscale-gemm-amd-agi

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Porting CK a8w8 fp8 block-scale GEMM kernels to FlyDSL on AMD MI355X (gfx950) fails parity because the hardware scaled-MFMA path only supports E8M0 power-of-two scales, while CK uses arbitrary fp32 per-block scales. This Skill provides a validated recipe to restore bit-level parity and then pick the correct shape-dependent performance lever. ## Core Features & Use Cases - Universal parity fix: Apply fp32 block scales in software after an unscaled fp8 MFMA, achieving err=0 / cos=1.0 against the CK oracle instead of silently rounding scales through E8M0 hardware. - Shape-dependent kernel selection: Choose between a 4-wave blockscale-preshuffle core (wide-N with XCD swizzle, or K-light with tile_n=256/tile_k=128) and an 8-wave ping-pong cluster core (narrow-N deep-K), with measured speedups from 1.05x to 1.45x. - Validated bottleneck diagnosis method: Use ablation, promote-free ceiling twins, roofline checks, and K-sweep phase decomposition to identify the true limiting factor before tuning. - Use Case: A profiled model shows a gemm_a8w8_blockscale hotspot on MI355X; apply this recipe to select the right FlyDSL core by M/N/K shape, pass parity first, then measure same-session A/B speedup versus CK. ## Quick Start Ask the agent to rewrite the CK fp8 a8w8 blockscale GEMM hotspot to FlyDSL on gfx950 using the software fp32 post-MFMA scale path and the shape-matched core recipe.

Frequently Asked Questions about flydsl_fp8_blockscale_gemm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I port a CK fp8 blockscale GEMM to FlyDSL on gfx950?

Keep the MFMA unscaled in fp8 and apply the arbitrary fp32 per-[128,128] block scales in software after the MFMA, using FlyDSL's 4-wave blockscale-preshuffle or 8-wave ping-pong cores. Never use the native E8M0 scaled-MFMA path, which silently rounds scales and fails parity.

Why does the FlyDSL scaled-MFMA path fail parity for blockscale GEMM?

The hardware scaled-MFMA scale is E8M0, which only represents powers of two, while CK a8w8 blockscale uses arbitrary fp32 scales per 128x128 block. Folding the fp32 scale into mfma_scale_*_f8f6f4 silently rounds it, so results diverge from the CK oracle.

Which FlyDSL core should I use for my GEMM shape?

Selection is by M/N/K shape: very-wide-N locality-limited shapes use the 4-wave core with xcd_swizzle=8, K-light grid-saturated shapes use the 4-wave core with tile_n=256/tile_k=128, and narrow-N deep-K shapes use the 8-wave ping-pong core with the cluster schedule at BLOCK_M=128, BLOCK_N=256.

Does XCD swizzling always improve GEMM performance on MI355X?

No, xcd_swizzle is shape-conditional. It is decisive on very-wide-N locality-limited shapes (about 1.23x from the xcd0 to xcd8 ablation) but a measured no-op on grid-saturated K-light and narrow-N tiny-C shapes, so treat it as an ablation axis rather than a default.

What speedup can I expect from this CK to FlyDSL rewrite?

Validated results range from about 1.05-1.10x on narrow-N deep-K shapes, 1.1-1.2x on wide-N shapes, up to 1.45x on K-light shapes. Correctness gates every candidate, and only same-session interleaved A/B ratios are quoted because GPUs drift about 5% run-to-run.

When should I stop tuning the kernel and look elsewhere?

Stop kernel-level tuning once the winner is within a couple percent of the ablation or promote-free ceiling-twin number. The remaining lever is E2E epilog fusion to cut the C round-trip, which is only visible in the full model graph, not in a standalone kernel.