skills.rest
yo-steven
yo-stevenCOMMUNITY
Explore all skills by this author →
GitHub

perf-host-optimization

Profile and optimize TensorRT-LLM host CPU overhead with line_profiler.

Updated May 23, 2026
Search Tags:#profiling#performance engineering#line_profiler#tensorRT-llm#host optimization#throughput tuning#CPU bottleneck
One-click install
npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill perf-host-optimization-yo-steven
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: perf-host-optimization
Source: https://github.com/yo-steven/skills-exploration-20260522/tree/main/skills/TensorRT-LLM/perf-host-optimization
Command: npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill perf-host-optimization-yo-steven

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill profiles and optimizes TensorRT-LLM host-side CPU overhead so inference throughput improves when the GPU is underutilized or when PyExecutor request handling becomes a bottleneck.

Core Features & Use Cases

  • Line-by-line host profiling with line_profiler: Uses TLLM_LINE_PROFILER_* environment variables to capture hotspots in PyTorch backend functions.
  • Bottleneck confirmation and targeted drill-down: Leverages perf-host-analysis results to prioritize likely functions (including NVTX-root-cause guided mapping) and forces deeper profiling when a line dominates (>80%) via drill-down targets.
  • Iterative profile-analyze-optimize-validate loop: Runs multiple rounds to apply low-risk optimizations first, mandates unit-test validation after each change, and re-profiles to verify both hotspot time and end-to-end benchmark metrics.
  • Host-aware tuning: Includes guidance on CPU affinity consistency (NUMA pinning) and workspace suffix conventions to keep iterative experiments comparable.

Quick Start

Run the profiling and optimization loop to reduce CPU bottlenecks in TensorRT-LLM host inference by enabling TLLM_LINE_PROFILER_ENABLED and setting TLLM_LINE_PROFILER_FUNCTIONS to the function you suspect is limiting throughput, such as tensorrt_llm._torch.pyexecutor.model_engine.PyTorchModelEngine._prepare_tp_inputs.

Frequently Asked Questions about perf-host-optimization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce CPU bottlenecks in TensorRT-LLM host inference?▼

Reduce TensorRT-LLM CPU bottlenecks by enabling TLLM_LINE_PROFILER_ENABLED and setting TLLM_LINE_PROFILER_FUNCTIONS to target suspected functions like PyTorchModelEngine._prepare_tp_inputs. Run iterative profile-analyze-optimize-validate rounds to cut host latency.

What is the best way to profile PyExecutor request handling bottlenecks?▼

Profile PyExecutor bottlenecks using line_profiler to capture line-by-line hotspots in PyTorch backend functions. Use drill-down targets to force deeper profiling when a single line dominates execution time, confirming root causes before optimizing.

Why does TensorRT-LLM throughput drop when GPU utilization is low?▼

TensorRT-LLM throughput drops when GPU utilization is low due to host-side CPU overhead in PyExecutor request handling. Profiling with line_profiler identifies these CPU bottlenecks, allowing targeted optimization to restore inference throughput.

Can I use NVTX root-cause mapping for TensorRT-LLM host optimization?▼

Yes, you can leverage prior perf-host-analysis NVTX root-cause mapping to prioritize likely bottleneck functions. This guides the line_profiler targeting, making it easier to identify and resolve specific host-side CPU overhead issues.

Do I need to run unit tests between TensorRT-LLM optimization rounds?▼

Yes, mandatory unit-test validation is required after each optimization change. This ensures low-risk optimizations maintain correctness before re-profiling to verify both hotspot time reduction and end-to-end benchmark improvements.

How to maintain CPU affinity consistency during TensorRT-LLM host profiling?▼

Maintain CPU affinity consistency during TensorRT-LLM profiling by applying NUMA pinning guidance and using workspace suffix conventions. This keeps iterative profiling experiments comparable across multiple optimization rounds.

Related Skills

Explore More Software Engineering Skills→
0xRayAI0xRayAI

performance-engineer

Profile system baselines and identify latency bottlenecks across application layers.

Official
Advanced
Quill-AgentQuill-Agent
3

tensorrt-llm

Optimize LLM inference on NVIDIA GPUs using TensorRT-LLM.

Community
Advanced
yo-stevenyo-steven

perf-host-analysis

Analyze nsys traces to detect host-side CPU overhead bottlenecks in TensorRT-LLM inference.

Community
Advanced
chflechfle
3

linux-performance-tuner

Profile Linux systems and generate sysctl, CPU governor, I/O scheduler configurations tailored to hardware and workload.

Community
Advanced