What problem does it solve?
Identifying the dominant performance bottleneck in a TensorRT-LLM serving deployment requires coordinating benchmarking, analytical speed-of-light projection, and multi-tool profiling (nsys, torch profiler, ncu) — a complex, error-prone process when done manually. This Skill drives the repo's perf-analyze workflow to produce a diagnosis report naming the single dominant bottleneck, without modifying the deployment.
Core Features & Use Cases
- Automated Benchmarking: Benchmarks trtllm-serve at a single concurrency point or a Pareto curve of points (tok/s/user vs tok/s/gpu) with configurable ISL/OSL, datasets, and request rates.
- Analytical SOL Projection: Derives a speed-of-light ceiling via the internal-perf-sol-analysis skill and correlates measured per-op times against it.
- Deep Kernel Profiling: Runs nsys, torch profiler, and a bounded ncu per-kernel deep dive on top kernels, then writes a performance report (Markdown + HTML) naming the single dominant bottleneck.
- Use Case: A user serving a model with trtllm-serve sees low throughput at concurrency 64. They run perf-analyze with their checkpoint and repo paths, and receive a report identifying the dominant bottleneck (e.g., KV-cache bound decode) with trace evidence and recommendations — nothing applied.
Quick Start
Ask the assistant to run perf-analyze on your trtllm-serve deployment by providing the model checkpoint path and TensorRT-LLM repo path, optionally with a concurrency list for Pareto-curve mode.
