catlass-op-perf-tune

Tune CATLASS kernel performance by adjusting TileShape, DispatchPolicy, Swizzle, and Kernel type parameters.

Updated Sep 15, 2026
One-click install
npx skills add https://github.com/WangWindow/CANN-BatchMatMulMaxsum --skill catlass-op-perf-tune-wangwindow
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: catlass-op-perf-tune
Source: https://github.com/WangWindow/CANN-BatchMatMulMaxsum/tree/main/.agents/skills/catlass-op-perf-tune
Command: npx skills add https://github.com/WangWindow/CANN-BatchMatMulMaxsum --skill catlass-op-perf-tune-wangwindow

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? CATLASS matmul and fused kernels on Ascend NPU often underperform due to suboptimal tiling, full-tile C HBM round-trips, and poor AIC/AIV pipeline overlap, and diagnosing these bottlenecks requires deep knowledge of profiler signals and hardware constraints. ## Core Features & Use Cases - Parameter Tuning Guidance: Explains the four tunable levers—Kernel type (C writeback strategy), DispatchPolicy (Pingpong vs Preload), L1/L0 TileShape, and BlockScheduler Swizzle—with capacity constraints and empirical starting points for fp16 and int8. - Bottleneck Diagnosis Tables: Maps msprof profiler symptoms (HBM bandwidth, MTE2 ratio, Cube utilization, AIV stalls) to root causes and prioritized fixes, including specialized tables for Linear Attention, KDA varlen, and FlashAttention stage tuning. - Single-Variable Tuning Discipline: Enforces one-change-at-a-time attribution with immediate rollback on regression, plus code diff patterns for switching Kernel writeback modes and DispatchPolicies. - Use Case: When a fused matmul_gelu kernel underperforms the torch baseline on large N shapes, use this Skill to identify the full-tile C HBM round-trip as the bottleneck and switch to a multi-stage rotating workspace kernel with PreloadAsyncWithCallback. ## Quick Start Ask the assistant to load the catlass-op-perf-tune skill and diagnose why my CATLASS matmul kernel is slower than the torch baseline using the profiler data.

Frequently Asked Questions about catlass-op-perf-tune

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I tune CATLASS kernel performance on Ascend NPU?

Adjust four parameters: Kernel type (C writeback strategy, the biggest lever), DispatchPolicy (Pingpong vs Preload scheduling), L1/L0 TileShape (buffer utilization), and BlockScheduler Swizzle (L2 hit rate). Change only one variable at a time and roll back immediately on regression.

What is the top optimization for CATLASS matmul kernels?

Eliminate the full-tile [M,N] fp32 C HBM round-trip used by MatmulActivation/MatmulEpilogue. Switch to a multi-stage rotating workspace kernel with MmadAtlasA2PreloadAsyncWithCallback so C stays L2-resident and AIC/AIV overlap at fine granularity.

What TileShape should I start with for fp16 CATLASS matmul?

For fp16, the empirical starting point is L1TileShape GemmShape<128,256,256> and L0TileShape GemmShape<128,256,64>, since the K-tile is capped around 256 by L1 capacity. For int8, K can reach 512 with L1<128,256,512>.

Why is my CATLASS kernel slow on large N shapes?

Large N slowness usually comes from the full [M,N] C tile round-tripping through HBM, shown by high HBM bandwidth, low L2 hit rate, and AIV waiting on C. Replace the Kernel with a multi-stage workspace variant before tuning TileShape or Swizzle.

Why does my custom CATLASS epilogue hang at runtime?

Runtime hangs in custom epilogues are typically caused by UB_STAGES exceeding the event ID limit of 8 per type. Reduce UB_STAGES, commonly to 1 for dual-input epilogues, and rebuild.

When should I stop tuning a CATLASS kernel?

Stop when Cube utilization reaches roughly 90 percent or higher, which indicates the kernel is near the roofline. Further gains are then limited by hardware constraints like fp16 L1 capacity, and remaining gaps should be documented with profiler evidence.