skills.rest
NVIDIA
NVIDIAOFFICIAL
Explore all skills by this author →
WebsiteGitHub

perf-optimization-casebook

Provides reusable TensorRT-LLM performance optimization precedents for classified bottlenecks.

14.5k|2.7k|Updated Aug 16, 2023
Search Tags:#CUDA#casebook#llm inference#tensorRT-llm#kernel fusion#Performance optimization#decision precedents
One-click install
npx skills add https://github.com/NVIDIA/TensorRT-LLM --skill perf-optimization-casebook
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: perf-optimization-casebook
Source: https://github.com/NVIDIA/TensorRT-LLM/tree/main/.claude/skills/perf-optimization-casebook
Command: npx skills add https://github.com/NVIDIA/TensorRT-LLM --skill perf-optimization-casebook

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

When optimizing TensorRT-LLM inference, engineers often guess at optimizations or reinvent approaches that have already been proven. This Skill supplies a curated casebook of past successful runtime, kernel, and communication optimizations recorded as decision precedents, so a classified bottleneck can be matched to prior art with known mechanisms, risks, and rollback paths.

Core Features & Use Cases

  • Precedent Matching: Free-text grep over case files plus optional canonical frontmatter ranking to find optimizations applicable to a given bottleneck class, model, hardware, and config.
  • Structured Case Schema: Every case records applicability signals, counter-signals, mechanism, generalization pattern, apply-via knob or specialist, accuracy risk, verification steps, and rollback triggers.
  • Controlled Vocabulary: Canonical tags, aliases, and a pattern registry (data/tags.yaml, aliases.yaml, patterns.yaml) keep matching consistent across runtime-execution, kernel-and-fusion, and communication families.
  • Use Case: After profiling classifies a decode bottleneck as launch-bound with many small kernels on SM90, consult the casebook to find precedents like PDL or piecewise CUDA-graph capture, adapt the knobs, and delegate implementation to the named specialist.

Quick Start

Ask the assistant to consult the perf-optimization-casebook for prior optimization precedents matching your classified TensorRT-LLM bottleneck, model, and hardware.

Frequently Asked Questions about perf-optimization-casebook

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find a prior TensorRT-LLM optimization for my bottleneck?▼

Classify the bottleneck first (compute, memory, launch, communication, sync, or host-overhead), then grep the case files in references/ with free-text terms like the model name or mechanism. Optionally rank hits using canonical frontmatter signals, then open only the winning case file.

What information does each optimization case record?▼

Each case records applicability signals, counter-signals, mechanism, a generalizes-to pattern, the exact config knob or specialist to apply it through, expected effect direction, accuracy risk (lossless, lossy, or mixed), verification steps, and rollback triggers.

Does the casebook run profiling or apply optimizations itself?▼

No. It is reference material only, consulted by coordinators like perf-analysis and perf-optimization. Implementation is always delegated to the specialist skill named in each case, and all numbers must come from real measurement.

Which optimization families does the casebook cover?▼

Three families: runtime/execution (overlap scheduler, CUDA graphs, chunked prefill, speculative decoding), kernel-and-fusion (op fusion, FP8/NVFP4 GEMM, MoE backends, sparse-MLA attention), and communication (DeepEP all-to-all, MNNVL AllReduce, symmetric memory).

When should I not apply a matched optimization case?▼

Check the case's Counter-signals field, which lists auto-disable regimes, degenerate shapes, negligible-share workloads, and unmet requirements. Also respect frontmatter interactions such as incompatible-with, and re-verify interface facts against your own TensorRT-LLM checkout.

How do I add a new optimization case to the casebook?▼

After a verified win, create references/<family>/<slug>.md from references/case-template.md with canonical frontmatter terms from data/tags.yaml, add a row to the family index, and register any new pattern in data/patterns.yaml with bidirectional case links.

Related Skills

Explore More Software Engineering Skills→
yo-stevenyo-steven

perf-nsight-compute-analysis

Classify SOL% bottlenecks and diagnose CUDA kernel underperformance from Nsight Compute output.

Community
Advanced
Open330Open330
1

benchmarking-ml-models

Benchmark ML models by measuring latency, throughput, memory, and accuracy.

Official
Intermediate
wshobsonwshobson
39.3k

quantized-export

Export promoted fine-tuned checkpoints as FP8, AWQ, GGUF, or merged safetensors with smoke-test validation.

Community
Advanced
alibabaalibaba
71

gpu-kernel-optimizer

Implement and optimize GPU kernels using PyTorch logic and DSLs.

Official
Advanced