cutedsl_megamoe

Updates and audits the vendored CuTeDSL MegaMoE kernel drop for FlashInfer expert-parallel MoE backends.

6.3k|1.4k|Updated Jul 22, 2023
One-click install
npx skills add https://github.com/flashinfer-ai/flashinfer --skill cutedsl-megamoe
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: cutedsl_megamoe
Source: https://github.com/flashinfer-ai/flashinfer/tree/main/flashinfer/moe_ep/kernel_src/cutedsl_megamoe
Command: npx skills add https://github.com/flashinfer-ai/flashinfer --skill cutedsl-megamoe

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, nvidia-cutlass-dsl.

What problem does it solve?

Keeping FlashInfer's expert-parallel MoE fast on Blackwell GPUs requires periodically re-vendoring NVIDIA's CuTeDSL MegaMoE kernel drop (NVFP4, MXFP8, BF16) without breaking the shim adapters, backend wrappers, or tuning profiles that depend on exact kernel symbols and launch signatures.

Core Features & Use Cases

  • Verbatim drop replacement workflow: Defines the exact procedure for replacing src/ with a new kernel-team drop while keeping all adaptation isolated in shim/.
  • Compatibility audit checklists: Provides symbol-by-symbol tables mapping shim imports to kernel source files so a renamed helper or changed launch signature breaks in exactly one auditable place.
  • Tuning and validation guidance: Pairs with TUNING.md for knob taxonomy, autotuning behavior, CuTe-DSL runtime version floors, and the multi-rank torchrun test command that confirms a drop works.
  • Use Case: When the NVIDIA kernel team publishes a new MegaMoE drop, follow this workflow to swap in the five kernel packages, audit the construct/launch signatures against the training drivers, and run the 4-GPU cutedsl test suite before merging.

Quick Start

Update the CuTeDSL MegaMoE kernel sources to the latest kernel-team drop and verify the shim adapters and tests still pass.

Frequently Asked Questions about cutedsl_megamoe

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I update the CuTeDSL MegaMoE kernel sources in FlashInfer?

Replace the five kernel packages under src/ verbatim with the new drop, then audit the shim imports and kernel construct/launch signatures against the new sources. Finish by running the multi-rank cutedsl test suite with torchrun on 4 Blackwell GPUs.

What is the difference between src/ and shim/ in the cutedsl_megamoe tree?

src/ is a verbatim copy of the NVIDIA kernel-team drop that must never be edited, while shim/ contains all FlashInfer adaptation code such as path bootstrap, symmetric-buffer wrappers, tuners, and autotuners. Backends import only the package __init__, never src/ directly.

Which CuTe-DSL runtime version do the MegaMoE kernels require?

nvidia-cutlass-dsl 4.5.2 is the performance floor: 4.5.0 fails at cute.compile, and 4.5.2 needs the MR!27 mainloop workaround while 4.5.3 and later run at full speed natively. The shim warns when an older runtime is detected.

Why do MegaMoE kernel updates break even when symbol names still exist?

The highest-churn surface is the kernel constructor and launch kwargs, where arguments change without names changing, so a symbol-existence grep misses it. The shim's _ensure_mega_compiled and _build_mega_runtime_kwargs must be re-audited against the kernel __init__ and __call__ signatures on every drop.

Can the SM90 and SM100 cutedsl kernel trees run in the same process?

No, both trees expose the same top-level module names, so bootstrap_paths raises if the sibling tree's modules are already imported. Use a separate process per architecture, which matches hardware reality since a process runs on either Hopper or Blackwell.