model-infer-superkernel

Configures SuperKernel operator binary fusion to optimize NPU inference decode performance.

Updated Sep 15, 2026
One-click install
npx skills add https://github.com/WangWindow/CANN-BatchMatMulMaxsum --skill model-infer-superkernel-wangwindow
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: model-infer-superkernel
Source: https://github.com/WangWindow/CANN-BatchMatMulMaxsum/tree/main/.agents/skills/model-infer-superkernel
Command: npx skills add https://github.com/WangWindow/CANN-BatchMatMulMaxsum --skill model-infer-superkernel-wangwindow

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Decode-stage inference on Ascend NPU suffers from high task scheduling overhead because each step launches many small operators. This Skill guides the adaptation of SuperKernel operator binary fusion, which merges operators within a marked scope to reduce kernel launch and scheduling costs. ## Core Features & Use Cases - Structured Four-Step Workflow: Analyzes model structure, determines fusion scope (Attention-only, MoE-only, full Decoder, or custom), modifies config.yaml and modeling code with superkernel_scope markers, and runs mandatory baseline-vs-optimized performance verification. - Constraint Enforcement: Validates prerequisites before acting—Atlas A3 hardware, ge_graph execution mode, PyTorch framework, and decode-only activation—while flagging dynamic shapes, Python control flow, and unsupported Tiling operators. - Use Case: A user running DeepSeek-R1 inference on Atlas A3 notices high decode latency. The Skill analyzes the MoE architecture, marks the MoE expert computation with superkernel_scope, enables enable_superkernel in config.yaml, and produces a superkernel_optimization_report.md with measured latency and throughput improvements. ## Quick Start Enable SuperKernel binary fusion for my model's decode stage and verify the performance improvement against a baseline.

Frequently Asked Questions about model-infer-superkernel

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I enable SuperKernel for NPU inference optimization?

Set exe_mode to ge_graph and enable_superkernel to True in config.yaml, then wrap the decode-stage code with the superkernel_scope context manager in the modeling file. Verify with baseline and optimized benchmark runs comparing decode latency and throughput.

What are the hardware and mode requirements for SuperKernel?

SuperKernel requires Atlas A3 hardware, the ge_graph execution mode, and the PyTorch framework. It only takes effect during the decode stage and is incompatible with eager and aclgraph modes.

Why does SuperKernel only benefit the decode stage?

Decode steps launch many small operators where scheduling overhead dominates, so binary fusion meaningfully reduces task dispatch cost. Prefill has large compute-heavy operators where scheduling overhead is a small fraction, and its dynamic input lengths cause SuperKernel to be disabled automatically.

How do I choose the SuperKernel scope for my model?

Analyze the model structure first, then choose between Attention-only, MoE-only, full Decoder layer, or a custom scope. First attempts should start with the smallest Attention scope, verify correctness and performance, then expand gradually.

Can SuperKernel fuse code with dynamic shapes or Python control flow?

No. Dynamic tensor shapes, data-dependent Python if/for control flow, and unsupported Tiling operators can prevent fusion. These must be identified during structure analysis and excluded or refactored before marking the scope.

Can I report SuperKernel optimization results without running benchmarks?

No. Performance verification is mandatory: you must run identical workloads with SuperKernel disabled and enabled, then compare decode latency, throughput, and output correctness. Fabricated improvement numbers without real measurements are not acceptable.