add-pallas-kernel

You can optimize TPU and GPU kernels using JAX's experimental Pallas library, which enables implementing, benchmarking, and autotuning high-performance computations. It enforces numerical safety and produces reference implementations with tuned tables for runtime selection.

Autotuning Pallas kernels automates the search for optimal block and tile sizes for specific hardware and shape regimes. This generates tuned tables used for runtime selection, ensuring optimal performance across varying tensor shapes.

Correctness and performance validation uses harnesses for value and gradient parity, numerics, and steady-state timing. This ensures your Pallas kernel matches the vanilla JAX implementation while achieving targeted acceleration performance.

Yes, Pallas is designed for developing new neural network layers requiring custom high-performance computation on accelerators. It provides the framework to build, test, and optimize the underlying kernel with consistent APIs.

Yes, the approach develops both vanilla JAX and Pallas kernel implementations with consistent APIs. This allows direct performance benchmarking and correctness comparisons between standard JAX operations and custom Pallas kernels.