Guides modification and review of KDA kernels in the Flash Linear Attention library.
npx skills add https://github.com/weichengz0616/fla --skill fla-kda-weichengz0616
Please help me install this Agent Skill. Skill: fla-kda Source: https://github.com/weichengz0616/fla/tree/main/.agents/skills/fla-kda Command: npx skills add https://github.com/weichengz0616/fla --skill fla-kda-weichengz0616
fla/ops/kda/** stay correct and consistent. ## Core Features & Use Cases - Public code map: Locates the exact functions for chunk forward, intra/inter kernels, WY recompute, backward passes, gate helpers, and FlashKDA/TileLang backends. - Gate mode contracts: Documents the pre-gated vs in-kernel gate modes, safe_gate constraints (-5 <= lower_bound < 0), and the midpoint-offset exponentiation invariant that keeps exp2 numerically safe. - Correctness checklist: Enumerates the test axes to cover (varlen, forward/backward, gate modes, MHA/GVA, CP paths, numerical extremes) before finishing a KDA behavior change. - Use Case: When modifying chunk_kda_fwd_kernel_intra_sub_chunk to change decay handling, use this Skill to verify the safe and non-safe intra paths stay consistent and to select the right tests. ## Quick Start Use the fla-kda skill to review my changes to the KDA safe-gate intra kernel and tell me which correctness axes I need to test.High-intent search queries and answers about installing and using this skill.
Start from the public code map covering chunk_kda, chunk_fwd, chunk_intra, wy_fast, and backward modules. Keep safe and non-safe intra paths consistent, then run the correctness checklist covering only the axes your change touches.
With safe_gate=True, activation is lower_bound * sigmoid(exp(A_log) * (g + dt_bias)) and requires use_gate_in_kernel=True with -5 <= lower_bound < 0. Without safe_gate, activation is -exp(A_log) * softplus(g + dt_bias).
Under lower_bound=-5, a 16-token sub-chunk can accumulate -80 in natural-log units, which would exceed exp2 safety limits if exponentiated directly. The midpoint offset b_gm = b_g - b_gn keeps each exponent operand within roughly half the sub-chunk range.
Cover dense and varlen layouts, forward and backward, all three gate modes, raw and post-sigmoid beta, use_qk_l2norm_in_kernel variants, MHA and GVA, and numerical extremes like lower_bound=-5, extreme A_log, and long-sequence cumulative decay.
No. Use the platform helpers from fla.utils such as device, device_platform, IS_NVIDIA, IS_AMD, and IS_INTEL instead of direct torch.cuda checks. If no helper covers your condition, add one in fla.utils first.
mindspore-aiImplement Triton-CUDA attention kernels with online Softmax for transformer workloads.
fla-orgManage KDA development tasks within the Flash Linear Attention framework.
fla-orgDefines contract-first design and coverage requirements for FLA kernel and numerical changes.
Benmore-StudioUpgrade Stripe API versions and SDKs across server, client, and mobile integrations.