add-jit-kernel

Name: add-jit-kernel
Availability: InStock
Author: moirai-internal

Guide users through adding a lightweight JIT CUDA kernel to SGLang's jit_kernel module.

3|Updated Jun 18, 2025

Search Tags:#optimization #kernel #developer guide #cuda #jit #sglang

Ready to run

npx skills add https://github.com/moirai-internal/sglang --skill add-jit-kernel-moirai-internal

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: add-jit-kernel
Source: https://github.com/moirai-internal/sglang/tree/main/.claude/skills/add-jit-kernel
Command: npx skills add https://github.com/moirai-internal/sglang --skill add-jit-kernel-moirai-internal

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of integrating custom CUDA kernels into the SGLang framework, enabling developers to extend its high-performance inference capabilities with their own optimized operations.

Core Features & Use Cases

JIT Kernel Integration: Provides a step-by-step guide for adding lightweight, Just-In-Time compiled CUDA kernels.
Abstraction Usage: Demonstrates the use of SGLang's provided C++ abstractions for safety, readability, and consistency.
Use Case: A developer needs to implement a novel attention mechanism or a custom activation function for a new LLM. This Skill guides them through creating and integrating that custom CUDA kernel into SGLang.

Quick Start

Follow the tutorial to implement a new CUDA kernel in jit_kernel/csrc/ and add its Python wrapper in jit_kernel/.

Frequently Asked Questions about add-jit-kernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add a custom CUDA kernel to SGLang for performance-critical operations?▼

This Skill guides you through adding a JIT CUDA kernel to SGLang by implementing C++ code in jit_kernel/csrc/ and adding Python wrappers in jit_kernel/. It leverages SGLang's provided abstractions for safety and consistency during integration.

What is a JIT CUDA kernel and when do I need one in SGLang?▼

A JIT CUDA kernel is a lightweight, Just-In-Time compiled piece of CUDA code integrated into SGLang. You need one when implementing novel attention mechanisms or custom activation functions for LLMs that require high-performance inference operations.

Can I use SGLang abstractions when writing custom CUDA kernels?▼

Yes, SGLang provides specific C++ abstractions designed for safety, readability, and consistency. This Skill demonstrates how to utilize these built-in abstractions while developing your custom JIT CUDA kernel integration.

How do I test and benchmark a new CUDA kernel added to SGLang?▼

Testing and benchmarking procedures are detailed within this Skill. After implementing your C++ kernel in jit_kernel/csrc/ and its Python wrapper, you follow the provided testing steps to ensure correctness and benchmarking to verify performance gains.

Does adding a custom kernel to SGLang require managing external dependencies?▼

No, this Skill streamlines the integration of lightweight JIT CUDA kernels directly into the SGLang framework without requiring external dependencies, focusing on extending high-performance inference capabilities seamlessly.