skills.rest
huggingface
huggingfaceOFFICIAL
Explore all skills by this author →
Website𝕏@huggingfaceGitHub

cpu-kernels

Writes, optimizes, and benchmarks C++ CPU kernels with AVX2/AVX512 SIMD intrinsics for Hugging Face kernels.

736|125|Updated Nov 29, 2024
Search Tags:#benchmarking#simd#kernel optimization#avx512#pyTorch#cpu kernels#brgemm
One-click install
npx skills add https://github.com/huggingface/kernels --skill cpu-kernels
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cpu-kernels
Source: https://github.com/huggingface/kernels/tree/main/kernel-builder/skills/cpu-kernels
Command: npx skills add https://github.com/huggingface/kernels --skill cpu-kernels

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, yaml, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Writing high-performance C++ CPU kernels with SIMD intrinsics is error-prone and slow to iterate on. This Skill provides an autonomous two-phase workflow — correctness first, then branching performance exploration — with deterministic scripts for analysis, validation, benchmarking, and profiling of x86 CPU kernels in the Hugging Face kernels ecosystem.

Core Features & Use Cases

  • Two-Phase Optimization Workflow: Phase 1 builds correct kernels tier by tier (generic ATen fallback → optional AVX2 → AVX512); Phase 2 runs a branching trial loop with backtracking until a target speedup or max trials is reached.
  • Deterministic Tooling: Scripts for op analysis (analyze_op.py), static validation (validate_cpu_kernel.py), correctness and speedup benchmarking (benchmark_cpu.py), perf stat profiling (cpu_profiler.py), and trial-tree management (trial_manager.py).
  • GEMM Acceleration Guidance: Covers quantized GEMM (INT4/NF4/FP4/FP8/MXFP4), Flash Attention, and MoE kernels using tinygemm for small M and at::native::cpublas::brgemm (oneDNN/AMX) for large M.
  • Use Case: Optimize an RMSNorm kernel for Intel Xeon — the agent analyzes the op, builds the kernel with kernel-builder, benchmarks against a PyTorch baseline, profiles with perf stat, and iterates trials until reaching the configured speedup target.

Quick Start

Ask the agent to write and optimize an AVX512 CPU kernel for a given PyTorch baseline such as RMSNorm, providing the baseline file and target shapes.

Frequently Asked Questions about cpu-kernels

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write an optimized AVX512 CPU kernel for PyTorch?▼

Start with a generic ATen fallback, then add an AVX512 implementation in a separate translation unit compiled with -mavx512f -mavx512bf16 and related flags. Use runtime dispatch via a cpu_features.hpp CPUID check, and validate correctness and speedup against a PyTorch baseline with the benchmark script.

How do I benchmark a custom C++ kernel against a PyTorch baseline?▼

Use benchmark_cpu.py with the baseline file, the installed kernel package name, and the package.function path. It checks numerical correctness with configurable tolerances and measures median latency via torch.utils.benchmark to report speedup.

When should I use brgemm versus tinygemm for CPU GEMM kernels?▼

Use tinygemm with hand-written _mm512_dpbf16_ps intrinsics for small M (M ≤ 4 for bf16) where brgemm overhead dominates. Use at::native::cpublas::brgemm for larger M, where oneDNN internally dispatches to AMX tile instructions on Intel Xeon 4th Gen and newer.

Why does my AVX512 kernel segfault or produce wrong results?▼

Common causes are aligned load intrinsics on unaligned data — always use _mm512_loadu_* — and missing tail handling when hidden_size is not divisible by the vector width. Also verify the dispatcher translation unit is compiled without -mavx flags and that CPUID checks include OS support via XCR0.

Does this workflow support quantized GEMM formats like INT4 or NF4?▼

Yes, it covers INT4, NF4, FP4, FP8, and MXFP4 quantized GEMM using a shared skeleton of nibble split, LUT lookup, and _mm512_dpbf16_ps accumulation. Weights must be pre-converted to a block-interleaved format by the framework, with dequantization fused per-forward or via brgemm.

What are the limitations of the CPU kernel optimization workflow?▼

It targets x86 CPUs with AVX2/AVX512 only and requires kernel-builder, PyTorch, and optionally perf stat for profiling. Only C++ kernel files and build.toml may be modified, and custom timing scripts are prohibited in favor of the provided benchmark tool.

Related Skills

Explore More Software Engineering Skills→
vipshopvipshop
1.2k

cutlass-cpp-kernel

Implement and optimize CUTLASS and CuTe C++ kernels with validation.

Official
Advanced
samuelbonifacio015samuelbonifacio015

llama-cpp

Run local GGUF inference with llama.cpp models from Hugging Face Hub.

Community
Advanced
GhostOf0daysGhostOf0days

hardware-software-codesign

Automate hardware-software codesign workflows for trading system latency optimization.

Community
Advanced
m0atm0at
1

triton-skill

Develop, debug, and autotune Triton and Gluon GPU kernels for ML workloads.

Community
Advanced