Guides development and optimization of MNN Metal backend kernels for LLM inference.
npx skills add https://github.com/alibaba/MNN --skill metal-optimize
Please help me install this Agent Skill. Skill: metal-optimize Source: https://github.com/alibaba/MNN/tree/main/skills/metal-optimize Command: npx skills add https://github.com/alibaba/MNN --skill metal-optimize
Developing and optimizing Metal GPU kernels in the MNN inference engine involves many silent-failure traps: stale build artifacts, pipeline cache hits, macro alias conflicts, memory aliasing in fused dispatches, and misleading profiling data. This Skill routes engineers to the right sub-document so kernel changes, operator fusion, and runtime scheduling work are done correctly and verified with proper A/B methodology.
Ask the assistant to help you add or optimize a Metal kernel in the MNN backend, describing your target model, quantization bit width, and whether the workload is prefill or decode.
High-intent search queries and answers about installing and using this skill.
Write the kernel as a C++ string literal in a *Shader.hpp file, create an Execution class implementing onResize and onEncode, and register it with REGISTER_METAL_OP_CREATOR. Follow the naming convention of operator_stage_variant_suffix and synchronize any new macro across shader, pipeline cache key, macro dictionary, and grid computation.
Focus on GEMV kernels which take about 70% of decode time. Documented techniques include deferred dequantization, split-K simdgroup pairing, and operator fusion of QKV and gate/up projections. Always verify with paired A/B runs since thermal drift can fake large gains.
Incremental make may not relink libMNN.dylib after editing shader strings in headers, and the Metal pipeline binary cache can load stale binaries. Touch the modified header, rebuild with make -B, delete mnn_cachefile.bin, and assert new symbols exist in the dylib before measuring.
Yes, since the causal detection became data-driven: models exporting a real mask tensor automatically disable causal-tri, causal-bound, and flash-attention paths. Standard causal models like Qwen and Llama use a scalar sentinel mask and keep all optimizations enabled.
Non-deterministic fused output usually indicates memory aliasing: a predecessor input was reused by the allocator as a successor output within one dispatch. The fix is alias detection with tensorsOverlap plus re-homing outputs to STATIC memory, or skipping fusion when re-homing fails.
No, profile build absolute numbers are artifacts because counter sample buffers inflate CPU encode cost roughly 20x and create fake GPU idle gaps. Use profile builds only for relative proportions, and validate any optimization with production-build paired A/B runs.
alibabaAdd new LLM models to MNN by downloading, analyzing, mapping, and exporting.
syhyaOptimize FlashInfer CUDA kernels for NVIDIA B200 using profiling and parity validation.
Kushal9889Diagnose complex technical issues through reproduction, hypothesis testing, and regression testing.
choice5346Run LLM inference on CPUs, Apple Silicon, and non-NVIDIA GPUs with llama.cpp.
