inference-performance

Name: inference-performance
Availability: InStock
Author: jpoindexter

Diagnose and optimize LLM inference performance metrics like TTFT and TPOT.

1|Updated Jun 9, 2026

Search Tags:#LLM #batching #performance diagnostics #inference performance #GPU sizing

Ready to run

npx skills add https://github.com/jpoindexter/design-and-ai-skills --skill inference-performance

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: inference-performance
Source: https://github.com/jpoindexter/design-and-ai-skills/tree/main/ai-engineering-skills/inference-performance
Command: npx skills add https://github.com/jpoindexter/design-and-ai-skills --skill inference-performance

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps diagnose and improve the performance of LLM inference in production environments, addressing issues like TTFT/TPOT/throughput, batching strategy, GPU sizing, and debugging low GPU utilization.

Core Features & Use Cases

Performance Diagnostics: Identify bottlenecks in LLM inference, such as prefill vs decode phase issues, TTFT spikes, and OOM errors.
Batching Strategies: Choose optimal batching strategies for LLM inference, considering prefill and decode phases.
GPU Sizing: Determine appropriate GPU sizes for serving LLMs based on performance metrics.
Use Case: For a company deploying a new LLM service, this Skill can help ensure that the service meets its latency and throughput requirements by optimizing the configuration and batching strategy.

Quick Start

Run the inference-performance skill to analyze the performance of your LLM inference service and optimize it for better throughput and lower latency.

Frequently Asked Questions about inference-performance

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I diagnose high TTFT and low throughput in LLM inference?▼

Diagnose LLM inference performance by isolating prefill versus decode phase bottlenecks. Analyze TTFT spikes and throughput drops to identify whether latency stems from initial prompt processing or token generation inefficiencies.

What is the best batching strategy for optimizing LLM inference latency?▼

Optimal batching strategies for LLM inference balance prefill and decode phases to maximize throughput. Choosing the right batching approach prevents GPU underutilization and minimizes TPOT without triggering OOM errors.

Why does my LLM inference service have low GPU utilization?▼

Low GPU utilization in LLM inference often indicates suboptimal batching strategies or incorrect GPU sizing. Debugging this requires analyzing prefill and decode phase metrics to ensure the hardware matches workload demands.

How do I determine GPU sizing for serving large language models in production?▼

Determine GPU sizing for LLM inference by evaluating performance metrics like TTFT, TPOT, and throughput targets. Matching GPU capacity to your specific batching strategy ensures production workloads meet latency requirements.

Can I use this approach to fix OOM errors during LLM inference batching?▼

Yes, analyzing prefill and decode phase dynamics helps resolve OOM errors during LLM inference batching. Adjusting batch sizes and GPU allocation based on performance diagnostics prevents memory exhaustion while maintaining throughput.