skills.rest
Romaosir
RomaosirCOMMUNITY
Explore all skills by this author →
GitHub

b200-moe-kernel-optimization

Optimize Mixture-of-Experts CUDA kernels for NVIDIA B200 with FP8/FP16 grouped-GEMM pipelines.

3|Updated May 14, 2026
Search Tags:#mixture of experts#cutlass#nc u profiling#cuda kernel optimization#fp8 correctness#nvidia b200#tcgen05
One-click install
npx skills add https://github.com/Romaosir/IF_Romao_kernel_optimize --skill b200-moe-kernel-optimization
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: b200-moe-kernel-optimization
Source: https://github.com/Romaosir/IF_Romao_kernel_optimize/tree/main/skills/b200_skill/b200-moe-kernel-optimization
Command: npx skills add https://github.com/Romaosir/IF_Romao_kernel_optimize --skill b200-moe-kernel-optimization

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps you optimize Mixture-of-Experts CUDA kernels for NVIDIA B200 (Blackwell, SM100) so you achieve major speedups while maintaining correctness, especially for FP8/FP16 grouped GEMM pipelines with routing, gather/scatter, SwiGLU, and Blackwell-specific primitives.

Core Features & Use Cases

  • Correctness-first optimization for MoE FP8: establishes a cuBLAS FP16 oracle baseline, uses strict failure-mode debugging for FP8 correctness, and avoids known-bad approaches documented in the dead-ends catalog.
  • B200-specific optimization ladder with measurable ROI: guides you through an ordered sequence (cuBLAS FP16 → CUTLASS FP8 grouped GEMM → zero-sync fast path → dual-tile dispatch → static compile → tcgen05), including plateau rules and regression discipline.
  • Backend selection + dispatch guardrails: covers CUTLASS vs tcgen05 decision points, T-dependent GEMM2 backend switching, and validation against silent fallbacks using NCU kernel-name checks.
  • Production-style team workflow: recommends role separation (Planner/Implementer/Profiler) and GPU isolation rules to keep measurements trustworthy across multi-round tuning.

Quick Start

Use the b200-moe-kernel-optimization skill to migrate your PyTorch Mixture-of-Experts reference to a B200-optimized CUDA implementation by following the optimization ladder and using cuBLAS FP16 as the correctness oracle on each change.

Frequently Asked Questions about b200-moe-kernel-optimization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize Mixture-of-Experts CUDA kernels for NVIDIA B200?▼

This Skill guides MoE kernel optimization for NVIDIA B200 by establishing a cuBLAS FP16 oracle baseline, applying strict failure-mode debugging, and avoiding known-bad approaches documented in a dead-ends catalog to ensure robust FP8 correctness.

What is the best way to ensure FP8 correctness when migrating MoE operators to Blackwell?▼

This Skill guides MoE kernel optimization for NVIDIA B200 by establishing a cuBLAS FP16 oracle baseline, applying strict failure-mode debugging, and avoiding known-bad approaches documented in a dead-ends catalog to ensure robust FP8 correctness.

How do I detect silent CUTLASS fallbacks during CUDA kernel optimization?▼

You can use CUTLASS for FP8 grouped GEMM and tcgen05 for Blackwell-specific primitives, applying T-dependent GEMM2 backend switching and dispatch guardrails to select the best backend based on your specific MoE operator dimensions.

When should I use CUTLASS versus tcgen05 for B200 MoE grouped GEMM?▼

You can use CUTLASS for FP8 grouped GEMM and tcgen05 for Blackwell-specific primitives, applying T-dependent GEMM2 backend switching and dispatch guardrails to select the best backend based on your specific MoE operator dimensions.

What workflow do I need to follow when tuning MoE kernels for FP8 speedups?▼

When tuning MoE kernels for FP8 speedups, follow a production-style team workflow with role separation like Planner, Implementer, and Profiler, apply GPU isolation rules, and use an ordered optimization ladder with strict regression discipline.

Does this MoE optimization approach handle routing and SwiGLU operations?▼

Yes, this MoE optimization approach handles routing, gather/scatter, and SwiGLU operations by converting the entire PyTorch reference into a high-performance FP8/FP16 grouped-GEMM pipeline tailored for the B200 architecture.

Related Skills

Explore More Software Engineering Skills→
Clay-HHKClay-HHK
2

deepspeed

Optimize distributed training with ZeRO stages, parallelism, and mixed precision.

Community
Advanced
vipshopvipshop
1.2k

cute-dsl-kernel

Implement and optimize CuTe DSL GPU kernels across NVIDIA architectures.

Official
Advanced
LeightonWangLeightonWang

triton-skills

Write, debug, and optimize Triton and Gluon GPU kernels.

Community
Advanced
yo-stevenyo-steven

cutile-python

Generate validated cuTile GPU kernels with float16 inputs and float32 accumulators.

Community
Advanced