skills.rest
jpoindexter
jpoindexterCOMMUNITY
Explore all skills by this author →
Website𝕏@microninkGitHub

inference-performance

Diagnose and optimize LLM inference performance metrics like TTFT and TPOT.

1|Updated Jun 9, 2026
Search Tags:#LLM#batching#performance diagnostics#inference performance#GPU sizing
One-click install
npx skills add https://github.com/jpoindexter/design-and-ai-skills --skill inference-performance
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: inference-performance
Source: https://github.com/jpoindexter/design-and-ai-skills/tree/main/ai-engineering-skills/inference-performance
Command: npx skills add https://github.com/jpoindexter/design-and-ai-skills --skill inference-performance

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps diagnose and improve the performance of LLM inference in production environments, addressing issues like TTFT/TPOT/throughput, batching strategy, GPU sizing, and debugging low GPU utilization.

Core Features & Use Cases

  • Performance Diagnostics: Identify bottlenecks in LLM inference, such as prefill vs decode phase issues, TTFT spikes, and OOM errors.
  • Batching Strategies: Choose optimal batching strategies for LLM inference, considering prefill and decode phases.
  • GPU Sizing: Determine appropriate GPU sizes for serving LLMs based on performance metrics.
  • Use Case: For a company deploying a new LLM service, this Skill can help ensure that the service meets its latency and throughput requirements by optimizing the configuration and batching strategy.

Quick Start

Run the inference-performance skill to analyze the performance of your LLM inference service and optimize it for better throughput and lower latency.

Frequently Asked Questions about inference-performance

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose high TTFT and low throughput in LLM inference?▼

Diagnose LLM inference performance by isolating prefill versus decode phase bottlenecks. Analyze TTFT spikes and throughput drops to identify whether latency stems from initial prompt processing or token generation inefficiencies.

What is the best batching strategy for optimizing LLM inference latency?▼

Optimal batching strategies for LLM inference balance prefill and decode phases to maximize throughput. Choosing the right batching approach prevents GPU underutilization and minimizes TPOT without triggering OOM errors.

Why does my LLM inference service have low GPU utilization?▼

Low GPU utilization in LLM inference often indicates suboptimal batching strategies or incorrect GPU sizing. Debugging this requires analyzing prefill and decode phase metrics to ensure the hardware matches workload demands.

How do I determine GPU sizing for serving large language models in production?▼

Determine GPU sizing for LLM inference by evaluating performance metrics like TTFT, TPOT, and throughput targets. Matching GPU capacity to your specific batching strategy ensures production workloads meet latency requirements.

Can I use this approach to fix OOM errors during LLM inference batching?▼

Yes, analyzing prefill and decode phase dynamics helps resolve OOM errors during LLM inference batching. Adjusting batch sizes and GPU allocation based on performance diagnostics prevents memory exhaustion while maintaining throughput.

Related Skills

timurgaleevtimurgaleev
5

benchmark

Detect performance regressions by comparing page load timings and resource metrics.

Community
Advanced
Ced3-hanCed3-han

awq-quantization

Automate 4-bit activation-aware quantization for large language models.

Community
Advanced
agentsopeagentsope
287

agentsop-vllm

Diagnose vLLM inference issues and tune deployment configurations for GPU workloads.

Community
Advanced
martinholovskymartinholovsky
45

model-quantization

Quantize AI models to 4-bit, 8-bit, and GGUF formats for deployment.

Community
Advanced