Write a Numerically Stable Kernel

What problem does it solve?

Many high-performance compute kernels produce incorrect results, NaNs/Infs, or large relative errors when run in low-precision dtypes (fp16, bf16) or on long accumulations. This Skill guides an agent to locate every accumulation and reduction path, classify numerical risks (overflow, catastrophic cancellation, accumulation error, subnormal underflow, inf propagation), and apply targeted, minimal-cost stabilizations so kernels remain correct and performant.

Core Features & Use Cases

• Risk enumeration & classification: Systematically list every place values are combined and label the numeric failure mode.
• Targeted mitigation patterns: Provide exact stabilized mathematical formulations (Welford, log-sum-exp, shift-by-max, fp32 accumulation) with dtypes annotated.
• Design tradeoffs & performance accounting: State whether an algorithm requires extra passes, increased registers, or different tensor core APIs, and quantify expected costs.
• Verification guidance: Include fp64 reference tests and adversarial input cases (large magnitudes, tiny values, all-equal inputs) to validate correctness.
• Use Cases: Stabilizing softmax/log-softmax/attention kernels for fp16, computing variance robustly in streaming pipelines, correcting long dot-product accumulation in mixed-precision kernels.

Quick Start

Stabilize this softmax kernel for fp16 by subtracting the per-row maximum, accumulating sums in fp32, emitting explicit dtype annotations at each step, and providing an fp64 reference test.