kernel-triton-writing

To write correct Triton kernels with mixed precision, you must enforce FP16/BF16 casting rules and use fp32 accumulators for transcendentals. This ensures numerical correctness while maintaining GPU kernel performance across deep learning compute patterns.

The best way to verify Triton GPU kernel correctness is to run a validation script against a PyTorch reference implementation. This requires exporting a kernel_fn, reference_fn, and get_inputs() to test real tensor shapes and dtypes for numerical and boundary correctness.

To implement safe masking and integer indexing in Triton kernels, apply proper masking for loads and stores, and use C-style semantics for integer operations like floor division (//) and modulo (%). This prevents out-of-bounds memory access during GPU kernel execution.

Yes, this workflow supports writing and verifying flash attention and tiled GEMM kernels with autotune in Triton. It provides specific guidance for these deep learning compute patterns alongside fused elementwise ops and reductions.

You should use Triton kernels when you explicitly need @triton.jit kernel code for fused deep learning operations rather than CUDA C++ or higher-level library calls. This approach is ideal when you must ensure numerical correctness for real tensor shapes and dtypes.