flydsl_fused_attention_backward

Authors a fused multi-GEMM FlyDSL attention backward kernel for CDNA3/CDNA4 AMD GPUs.

178|52|Updated Jul 30, 2025
One-click install
npx skills add https://github.com/AMD-AGI/GEAK --skill flydsl-fused-attention-backward-amd-agi
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: flydsl_fused_attention_backward
Source: https://github.com/AMD-AGI/GEAK/tree/main/perf_knowledge/expert_skills/skills/flydsl_fused_attention_backward
Command: npx skills add https://github.com/AMD-AGI/GEAK --skill flydsl-fused-attention-backward-amd-agi

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Training and fine-tuning runs on AMD Instinct GPUs (gfx942/gfx950) often bottleneck on attention backward kernels when the vendor fast path cannot dispatch natively — for example when head dims are asymmetric (hdim_q != hdim_v) and the tuned asm/CK backward pads or falls back to a slow generic path. This Skill provides a regulated, step-by-step procedure for authoring a fused multi-GEMM FlyDSL backward kernel that avoids padding overhead and register spills. ## Core Features & Use Cases - Structural kernel design guidance: Prescribes four key decisions — fusing all GEMMs into one kernel, operand-swapped MFMA fragment orientation, KV-outer looping to avoid atomics, and wave-count selection against the 256 arch-VGPR register cap. - Gated bring-up procedure: Enforces a dependency-ordered ladder (fp32 reference, harness with ISA probe, preprocess kernel, main kernel, incremental optimization, head-to-head benchmark) with parity gates of cos >= 0.999 on every gradient. - Documented dead ends: Lists measured anti-patterns (oversized KV tiles, dQ atomic fan-out, barrier folding) so engineers do not re-derive failed optimizations. - Use Case: An MLA attention backward with HD_QK=192/HD_V=128 on gfx942 reaches 1.38x speedup per unit of useful work versus the padded-ASM baseline by following this recipe. ## Quick Start Ask the agent to author a fused FlyDSL attention backward kernel for your asymmetric-head-dim training workload on gfx942, following the gated bring-up procedure with fp32 parity validation.

Frequently Asked Questions about flydsl_fused_attention_backward

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write a fused attention backward kernel for AMD MI300X?

Follow the gated procedure: build an fp32 reference with fixed causal convention, create a benchmark harness with an ISA probe, write a preprocess kernel computing D = rowsum(dO * O), then the main fused kernel with three LDS tiles and KV-outer looping. Gate each step on gradient parity (cos >= 0.999) and register counts from the ISA dump.

When should I author a custom FlyDSL attention backward kernel?

Author one when the vendor fast path cannot dispatch natively for your shape — typically asymmetric head dims (hdim_q != hdim_v) that force padding, or GQA/MQA variants with no tuned backward. Do not author one when a tuned asm/CK backward already dispatches natively, since that path is strong and hard to beat.

Does this kernel recipe work on gfx950 as well as gfx942?

The structural recipe applies to both gfx942 and gfx950, but the wave-count choice must be re-derived per generation because gfx950 has a larger register file. The measured 1.38x speedup is gfx942-specific and includes a padding tax that does not exist on gfx950.

Why is my attention backward kernel slower despite fewer instructions?

Instruction count reductions often lose to register spills because the fused backward carries multiple long-lived operand sets against a fixed 256 arch-VGPR cap. Check the ISA for v_accvgpr move counts — spills can relocate into the accumulator file without appearing as scratch memory.

What are the limitations of the fused backward approach?

It is training-only and does not apply to inference serving paths. Symmetric head dims are usually not worth authoring since the tuned asm backward dispatches natively, and non-causal attention removes the cost gradient that the longest-job-first grid ordering exploits.