extract-kernel-definitions

Extract kernel definitions from SGLang models by running the extraction tool with a model name. It identifies kernel types and parameters, then saves the output as FlashInfer-compatible JSON files into the flashinfer_trace/definitions directory.

You can generate FlashInfer definition JSONs for MLA, GQA, RMSNorm, GEMM, and MoE kernels. The tool parses model configs to identify these kernel schemas and aligns parameters using sgl-cookbook serving configurations.

To avoid duplicate kernel definitions across SGLang models, the extraction skill uses sgl-cookbook serving configurations to align parameters and automatically deduplicates schemas across multiple TP and EP configurations during generation.

Yes, you can generate kernel definitions for multiple TP and EP configurations in SGLang. The tool creates multiple variants from serving configurations and deduplicates the extracted schemas across different models automatically.

You need to parse the model config to extract kernel parameters and types. The tool uses sgl-cookbook serving configurations to align these parameters, ensuring accurate schemas and avoiding duplicates when outputting the definition JSONs.

FlashInfer kernel definitions may lack reference implementations if they are unavailable during extraction. The tool saves definition JSON files with naming conventions and includes reference implementations for validation only when they are available in the source model.