skills.rest
moirai-internal
moirai-internalOFFICIAL
Explore all skills by this author →
GitHub

add-jit-kernel

Guide users through adding a lightweight JIT CUDA kernel to SGLang's jit_kernel module.

3|Updated Jun 18, 2025
Search Tags:#optimization#kernel#developer guide#cuda#jit#sglang
One-click install
npx skills add https://github.com/moirai-internal/sglang --skill add-jit-kernel-moirai-internal
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: add-jit-kernel
Source: https://github.com/moirai-internal/sglang/tree/main/.claude/skills/add-jit-kernel
Command: npx skills add https://github.com/moirai-internal/sglang --skill add-jit-kernel-moirai-internal

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of integrating custom CUDA kernels into the SGLang framework, enabling developers to extend its high-performance inference capabilities with their own optimized operations.

Core Features & Use Cases

  • JIT Kernel Integration: Provides a step-by-step guide for adding lightweight, Just-In-Time compiled CUDA kernels.
  • Abstraction Usage: Demonstrates the use of SGLang's provided C++ abstractions for safety, readability, and consistency.
  • Use Case: A developer needs to implement a novel attention mechanism or a custom activation function for a new LLM. This Skill guides them through creating and integrating that custom CUDA kernel into SGLang.

Quick Start

Follow the tutorial to implement a new CUDA kernel in jit_kernel/csrc/ and add its Python wrapper in jit_kernel/.

Frequently Asked Questions about add-jit-kernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a custom CUDA kernel to SGLang for performance-critical operations?▼

This Skill guides you through adding a JIT CUDA kernel to SGLang by implementing C++ code in jit_kernel/csrc/ and adding Python wrappers in jit_kernel/. It leverages SGLang's provided abstractions for safety and consistency during integration.

What is a JIT CUDA kernel and when do I need one in SGLang?▼

A JIT CUDA kernel is a lightweight, Just-In-Time compiled piece of CUDA code integrated into SGLang. You need one when implementing novel attention mechanisms or custom activation functions for LLMs that require high-performance inference operations.

Can I use SGLang abstractions when writing custom CUDA kernels?▼

Yes, SGLang provides specific C++ abstractions designed for safety, readability, and consistency. This Skill demonstrates how to utilize these built-in abstractions while developing your custom JIT CUDA kernel integration.

How do I test and benchmark a new CUDA kernel added to SGLang?▼

Testing and benchmarking procedures are detailed within this Skill. After implementing your C++ kernel in jit_kernel/csrc/ and its Python wrapper, you follow the provided testing steps to ensure correctness and benchmarking to verify performance gains.

Does adding a custom kernel to SGLang require managing external dependencies?▼

No, this Skill streamlines the integration of lightweight JIT CUDA kernels directly into the SGLang framework without requiring external dependencies, focusing on extending high-performance inference capabilities seamlessly.

Related Skills

Explore More Software Engineering Skills→
sgl-projectsgl-project
31.6k

add-jit-kernel

Guide adding lightweight JIT CUDA kernels to the sglang framework.

Official
Advanced
scottgl9scottgl9
12

add-jit-kernel

Integrate custom CUDA JIT kernels into the sglang framework with Python wrappers.

Community
Advanced
spacecat2002spacecat2002

debug-cuda-crash

Diagnose CUDA crashes in SGLang by logging tensors at kernel API boundaries.

Community
Advanced
mindspore-aimindspore-ai
258

triton-cuda-examples-torch

Integrate Triton CUDA kernels with PyTorch models for custom operators.

Official
Intermediate