flash-attn-training

Add a sibling attention function returning the same shape as the manual chain, gate it behind a per-trainer flag defaulting to exact, add a supports_op probe at real shapes, a CPU f32 parity rung, a VRAM model branch, and a drift A/B measurement. The skill provides the full checklist per trainer.

Paired measurements on the DiT show flash is about 8.5% slower per token at equal graph shape; its win is the larger crop it affords, not per-token speed. The deficit is concentrated in the cross-attention backward kernel, which recomputes shared tiles.

Yes, the fused ops accept any additive F16 mask, GQA with Nkv less than Nh at batch 1, cross-attention with S_kv different from S, and non-contiguous q/k/v views. Causal masking uses one triangular minus-INF mask and skips fully masked tiles.

The backend scheduler registers CPU alongside CUDA, so an unsupported op is silently split onto the CPU instead of erroring. The contract requires a supports_op probe at the run's real shapes that aborts with a named error when the op is unsupported.

The default precision resolves to TF32 on sm_80 and newer because op_params zero-init maps to GGML_PREC_DEFAULT; GGML_PREC_F32 selects the scalar f32 kernels. Run logs must record the resolved precision, not just the requested mode.

Keep it off when a trainer uses a frozen KV prefix (the fused kernel cannot take the rectangular mask), when short crops already fit in memory since flash shows no benefit there, and until ear validation confirms adapter quality.