skills.rest
wikimedia
wikimediaOFFICIAL
Explore all skills by this author →
WebsiteGitHub

optimize-model

Identifies CPU/GPU bottlenecks in KServe-hosted Python models and outputs prioritized fix diffs.

4|Updated May 12, 2021
Search Tags:#kubernetes#model optimization#vllm#rocm#inference latency#kserve#cpu gpu tuning
One-click install
npx skills add https://github.com/wikimedia/machinelearning-liftwing-inference-services --skill optimize-model-wikimedia
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: optimize-model
Source: https://github.com/wikimedia/machinelearning-liftwing-inference-services/tree/main/.claude/skills/optimize-model
Command: npx skills add https://github.com/wikimedia/machinelearning-liftwing-inference-services --skill optimize-model-wikimedia

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you find and fix performance bottlenecks in a Python-based model server and its KServe deployment so you can improve inference throughput and latency on CPU or GPU (including AMD MI300X with ROCm and vLLM).

Core Features & Use Cases

  • End-to-end performance analysis across model code, dependencies, deployment chart values, and Kubernetes resource settings.
  • Backend-specific optimization guidance for vLLM (including AsyncLLMEngine, batching knobs, caching, and tensor parallelism) and ROCm/MI300X tuning (including AITER/flash-attention alignment and communication/shared-memory considerations).
  • Actionable remediation report that groups findings by layer (backend, external dependencies, KServe, Python, K8s) with severity, rationale, and concrete configuration/code diffs.

Use case example: If your team is seeing high latency or low throughput for a GPU-hosted model in production, use this Skill to pinpoint whether the bottleneck is vLLM configuration (batching, max sequence length, GPU memory utilization), ROCm environment variables, Python preprocessing blocking calls, or insufficient KServe/K8s worker and resource settings.

Quick Start

Ask the assistant to analyze the performance of your model server by providing the model name you want optimized (for example: policy_violation).

Frequently Asked Questions about optimize-model

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize model inference latency on KServe?▼

To optimize model inference latency on KServe, analyze your Python model server code, dependencies, and Helm chart deployment configuration to identify CPU and GPU performance bottlenecks. This process yields a prioritized report with concrete configuration and code diffs to improve throughput.

Why does my vLLM inference service have low throughput?▼

Low throughput in your vLLM inference service may stem from suboptimal vLLM configuration like batching knobs, max sequence length, GPU memory utilization, or insufficient KServe and Kubernetes worker resource settings. Analyzing these layers helps pinpoint the exact bottleneck.

Can I tune ROCm and MI300X environments for better model performance?▼

Yes, you can tune ROCm and MI300X environments for better model performance by aligning AITER and flash-attention configurations and optimizing communication and shared-memory considerations. The analysis compares your setup against upstream ROCm best-practice guides.

What is the best way to find CPU and GPU bottlenecks in a Kubernetes model deployment?▼

The best way to find CPU and GPU bottlenecks in a Kubernetes model deployment is to perform an end-to-end performance analysis across your model code, external dependencies, KServe, Python, and K8s resource settings. This approach groups findings by layer with severity and rationale.

Does this performance analysis work with CPU-only inference services?▼

Yes, this performance analysis works with CPU-only inference services as well as GPU inference services. It identifies performance bottlenecks by analyzing the model server implementation, dependencies, and Kubernetes resource settings to improve latency and throughput.

Related Skills

Explore More Software Engineering Skills→
sarimofficialsarimofficial

Helm Chart Generator

Generate Helm charts for multi-environment Kubernetes deployments of a full-stack Todo app.

Community
Advanced
nicholasgriffintnnicholasgriffintn
2

performance-optimisation

Identify and fix performance bottlenecks across frontend, backend, and database layers.

Community
Basic
ROCmROCm
260

build-rocm-image

Build a ROCm development Docker image on a remote host via SSH.

Official
Advanced
danmartinez78danmartinez78

serving-llms-vllm

Deploy vLLM servers with OpenAI-compatible endpoints and quantization.

Community
Advanced