add-jit-kernel

Name: add-jit-kernel
Availability: InStock
Author: sqjian

Automate adding lightweight JIT CUDA kernels to SGLang's jit_kernel module.

Updated May 14, 2026

Search Tags:#gpu #kernel #tutorial #cuda #sglang #jit-kernel

Ready to run

npx skills add https://github.com/sqjian/sglang --skill add-jit-kernel-sqjian

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: add-jit-kernel
Source: https://github.com/sqjian/sglang/tree/main/.claude/skills/add-jit-kernel
Command: npx skills add https://github.com/sqjian/sglang --skill add-jit-kernel-sqjian

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Step-by-step guide to extend SGLang with a new lightweight JIT CUDA kernel using the jit_kernel module, enabling rapid iteration and deployment.

Core Features & Use Cases

Create a new CUDA kernel under jit_kernel/csrc/ and a Python wrapper to expose it to the runtime.
Leverage TensorMatcher for robust input validation, device checks, and symbolic sizes.
Use the provided vectorization primitives (AlignedVector, LaunchKernel) to maximize bandwidth and ensure safe launches.
Write unit tests and benchmarks to validate correctness and measure performance on CUDA devices.

Quick Start

Follow the guide to implement a new JIT CUDA kernel by adding the kernel C++/CUDA source, Python wrapper, tests, and benchmarks, then run the provided CI-style tests to validate correctness.

Frequently Asked Questions about add-jit-kernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add a custom JIT CUDA kernel to SGLang?▼

To add a custom JIT CUDA kernel to SGLang, you implement the kernel source under jit_kernel/csrc/, create a Python wrapper for runtime exposure, and write corresponding tests and benchmarks. This ensures correct integration and rapid iteration.

What is the best way to validate tensor inputs and ensure safe kernel launches in SGLang?▼

The best way to validate tensor inputs and ensure safe kernel launches in SGLang is by using the TensorMatcher for input validation and device checks, combined with LaunchKernel for deterministic launches and AlignedVector for vectorized memory access.

Can I use SGLang's jit_kernel module to add a CUDA kernel that relies on large external dependencies?▼

No, the SGLang jit_kernel module is specifically designed for adding lightweight JIT CUDA kernels that do not rely on large dependencies, ensuring fast iteration and deployment without heavy external library requirements.

How does the SGLang JIT kernel workflow connect C++ source code with Python bindings?▼

The SGLang JIT kernel workflow connects C++ and Python by implementing the kernel logic in C++/CUDA source files and exposing it to the runtime through a Python wrapper, structuring the process across C++, Python, and tests.

Do I need to write benchmarks when extending SGLang with a new CUDA kernel?▼

Yes, you need to write benchmarks and unit tests when extending SGLang with a new CUDA kernel. This validates correctness and measures performance on CUDA devices, running CI-style tests to ensure quality.

When should I use AlignedVector and LaunchKernel in my SGLang JIT kernel?▼

You should use AlignedVector and LaunchKernel in your SGLang JIT kernel to maximize memory bandwidth and ensure safe, deterministic launches. These primitives provide vectorized memory access and structured execution.