skills.rest
vipshop
vipshopOFFICIAL
Explore all skills by this author →
WebsiteGitHub

triton-kernel

Write optimized Triton GPU kernels for deep learning operations.

1.3k|80|Updated Jun 12, 2025
Search Tags:#Optimization#CUDA#flash attention#matmul#GPU kernel#Triton#Deep Learning
One-click install
npx skills add https://github.com/vipshop/cache-dit --skill triton-kernel
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: triton-kernel
Source: https://github.com/vipshop/cache-dit/tree/main/.github/skills/triton-kernel
Command: npx skills add https://github.com/vipshop/cache-dit --skill triton-kernel

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires triton, torch, and includes references (resource) components.

What problem does it solve?

Writing high-performance GPU kernels for deep learning operations requires deep knowledge of memory hierarchies, tensor cores, and hardware-specific tuning, which most developers lack.

Core Features & Use Cases

  • Core Kernel Patterns: Provides verified patterns for masking, block sizing, FP32 accumulation, stride-based addressing, and autotune configuration for any Triton kernel.
  • Specialized Kernel Guides: Covers FlashAttention v2 with online softmax, persistent warp-specialized matmul with TMA, fused LayerNorm/RMSNorm, quantized block-scaled GEMM, and memory-efficient dropout via Philox PRNG.
  • Performance Diagnosis: Includes bottleneck classification (memory-bound, compute-bound, underutilized) with NCU metrics and hardware reference tables for A100, H100, and consumer GPUs.
  • Use Case: When implementing a fused attention kernel for a custom transformer, read the FlashAttention guide for the online softmax loop, then use the dynamic launcher guide to select tile sizes based on sequence length and dtype.

Quick Start

Write a Triton kernel for fused softmax over rows of a matrix and benchmark it against the PyTorch baseline.

Frequently Asked Questions about triton-kernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write a FlashAttention kernel in Triton?▼

Use a 2D grid of query blocks and batch-head pairs, maintain running softmax statistics (m, l, acc) in FP32 registers, and iterate over K/V blocks with online softmax updates. Apply sm_scale after tl.dot and cast p to the V tile dtype before the second dot product.

How to choose block sizes and num_warps for Triton kernels?▼

Select BLOCK_M based on query length (16 for decode, 128 for long prefill) and BLOCK_N based on KV length, capping both at 64 when HEAD_DIM exceeds 128. Use num_warps=4 by default, 8 for large 128x128 tiles, and 2 for decode paths.

Does Triton support FP8 and FP4 quantized matmul?▼

Triton supports block-scaled quantized matmul via tl.dot_scaled on SM100+/CDNA4 hardware for mxfp4, mxfp8, and nvfp4 formats. On older hardware, use a dequantize fallback that unpacks INT4/FP4 values, multiplies by scale tiles, and accumulates in FP32.

Why is my Triton kernel slower than expected?▼

Profile with ncu to classify the bottleneck: DRAM throughput above 60% indicates memory-bound (fix with fusion and swizzle), tensor core utilization above 60% indicates compute-bound (use persistent kernels), and both below 60% indicates register pressure or low occupancy.

What GPUs are supported for Triton kernel development?▼

Triton kernels target NVIDIA GPUs with SM70+ (Volta and newer) and AMD CDNA2+ architectures. TMA descriptors and warp specialization require SM90+ (Hopper), while tl.dot_scaled requires SM100+ or CDNA4.

Related Skills

Explore More Software Engineering Skills→
spacecat2002spacecat2002

debug-cuda-crash

Diagnose CUDA crashes in SGLang by logging tensors at kernel API boundaries.

Community
Advanced
big4council-progbig4council-prog

serving-llms-vllm

Deploy high-throughput LLM inference endpoints with vLLM and OpenAI-compatible APIs.

Community
Intermediate
alibabaalibaba
71

gpu-kernel-bottleneck-analysis

Analyzes GPU kernel performance using Roofline models and profiling data.

Official
Advanced
togethercomputertogethercomputer

diffusion-kernel

Build, test, and profile JIT CUDA and Triton kernels for SGLang diffusion models.

Official
Advanced