skills.rest
cloudrift-ai
cloudrift-aiOFFICIAL
Explore all skills by this author →
GitHub

tune-model

Autotune compiled LLM kernels and generate root-cause performance findings reports.

75|8|Updated Aug 2, 2025
Search Tags:#performance analysis#llm tuning#autotuning#vllm serving#kernel benchmarking#cuda profiling#emmy compiler
One-click install
npx skills add https://github.com/cloudrift-ai/emmy --skill tune-model
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tune-model
Source: https://github.com/cloudrift-ai/emmy/tree/main/.claude/skills/tune-model
Command: npx skills add https://github.com/cloudrift-ai/emmy --skill tune-model

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill eliminates the tedious, error-prone manual work of profiling compiled LLM kernels, benchmarking against PyTorch eager and torch.compile, and diagnosing root causes of performance gaps. Without it, developers spend hours sifting through NCU profiles, tune databases, and emitted CUDA to pinpoint why kernels underperform, and struggle to produce reproducible, actionable findings reports.

Core Features & Use Cases

  • Automated Clean Autotuning: Runs reproducible, cache-free autotuning passes for single layers or full LLM models, with support for dynamic sequence length shapes to produce deployable masked-tile kernels.
  • Comprehensive Benchmarking: Generates end-to-end and per-kernel benchmark tables comparing emmy performance against PyTorch eager and torch.compile, plus optional serving A/B benchmarks against vanilla vLLM for servable embedding models.
  • Root-Cause Triage: Automatically classifies underperforming kernels into four failure categories (search shortfall, tier/optimization lockout, codegen quality, bench failures) with supporting evidence from NCU metrics, tune-DB rows, and emitted CUDA code.
  • Structured Findings Reports: Produces standardized, reproducible reports saved to the plans/ directory, including repro commands, suggested fixes, and workflow retrospective notes for continuous improvement of the emmy compiler.
  • Use Case Example: A developer optimizing the Qwen/Qwen3-Embedding-0.6B model can run a single workflow to identify that a specific attention kernel is slow due to a tensor core eligibility gate, get a pinned repro command, and receive a suggested knob adjustment to unlock tensor core performance.

Quick Start

Use the tune-model skill to autotune the first layer of the Qwen/Qwen3-Embedding-0.6B model, benchmark its kernels against PyTorch eager and torch.compile, and generate a structured performance findings report saved to the plans/ directory.

Frequently Asked Questions about tune-model

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate CUDA profiling for LLM kernels underperforming against PyTorch eager?▼

Automate CUDA profiling by running reproducible autotuning passes that benchmark LLM kernels against PyTorch eager and torch.compile, generating structured findings reports with NCU metrics and root-cause triage to eliminate manual performance debugging work.

Why does my compiled LLM kernel underperform during vLLM serving A/B testing?▼

Underperforming kernels during vLLM serving A/B testing are automatically classified into failure categories like search shortfall, tier lockout, or codegen quality issues, using supporting evidence from NCU metrics and tune-DB rows to pinpoint the root cause.

Do I need a CUDA GPU with ncu perf counter access to autotune LLM kernels?▼

Yes, you need a CUDA GPU with ncu perf counter access to run the autotuning and benchmarking workflows, which use the emmy CLI's built-in commands for all measurements and root-cause performance analysis.

What's the best way to benchmark compiled LLM models against torch.compile?▼

Generate comprehensive end-to-end and per-kernel benchmark tables comparing emmy performance against PyTorch eager and torch.compile, with optional serving A/B benchmarks against vanilla vLLM for servable embedding models.

How do I generate reproducible performance findings reports for LLM compiler development?▼

Produce standardized findings reports saved to the plans directory, including repro commands, suggested fixes, NCU metrics, and workflow retrospective notes to ensure continuous improvement of the emmy compiler.

Can I autotune dynamic sequence length shapes for LLM embedding models?▼

Yes, run reproducible autotuning passes for single layers or full LLM models with support for dynamic sequence length shapes, producing deployable masked-tile kernels for models like Qwen/Qwen3-Embedding-0.6B.

Related Skills

Explore More Software Engineering Skills→
igamenovoerigamenovoer
7

pixi-make-cu-build-env

Configure CUDA build environments for Pixi projects with CMake and nvcc.

Community
Advanced
guan404mingguan404ming
1

tvm-dev

Resolve TODOs and issues in the TVM relax folder with minimal Python repros.

Community
Intermediate
RachasumanthRachasumanth

inference-deploy

Deploy trained machine learning models for inference using vLLM, TGI, Ollama, and llama.cpp.

Community
Advanced
RomaosirRomaosir
3

ncu-cuda-profiling

Automate Nsight Compute profiling and classify CUDA kernel bottlenecks.

Community
Advanced