skills.rest
BBuf
BBufCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llm-serving-auto-benchmark

Benchmark LLM serving frameworks across SGLang, vLLM, and TensorRT-LLM.

721|65|Updated Apr 1, 2026
Search Tags:#llm#benchmarking#config-validation#serving#cross-framework#benchmarking-tools
One-click install
npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill llm-serving-auto-benchmark
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-serving-auto-benchmark
Source: https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/llm-serving-auto-benchmark
Command: npx skills add https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS --skill llm-serving-auto-benchmark

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Cross-framework benchmarking of LLM serving stacks identifies the best deployment for a given model and workload.

Core Features & Use Cases

  • Framework-agnostic evaluation across SGLang, vLLM, and TensorRT-LLM on identical workloads.
  • Config-driven workflow that records framework versions, help outputs, server/benchmark commands, and artifact paths for fair comparisons.
  • Produces a cross-framework summary with best commands per framework and ranked results.

Quick Start

Run a cross-framework benchmark plan using the cookbook configs and validator tools to generate a normalized results report.

Frequently Asked Questions about llm-serving-auto-benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark LLM serving performance across vLLM, SGLang, and TensorRT-LLM?▼

Cross-framework benchmarking of LLM serving evaluates vLLM, SGLang, and TensorRT-LLM under identical workloads and SLA constraints. It records framework versions and artifact paths to produce a normalized report ranking the best deployment per model.

What is the best way to compare vLLM vs SGLang vs TensorRT-LLM serving latency?▼

The best way to compare serving latency across frameworks is using a config-driven benchmarking tool that evaluates vLLM, SGLang, and TensorRT-LLM under identical workloads. This ensures a fair comparison by recording versions and generating normalized, ranked results.

Can I evaluate LLM serving frameworks under identical workloads and SLA constraints?▼

Yes, you can evaluate LLM serving frameworks under identical workloads and SLA constraints. This config-driven workflow ensures fair comparisons by recording framework versions, server commands, and artifact paths, then generating a cross-framework summary with best commands per framework.

How do I generate a normalized benchmarking report for different LLM serving stacks?▼

You generate a normalized benchmarking report by running a cross-framework evaluation across SGLang, vLLM, and TensorRT-LLM. The config-driven workflow records framework versions and help outputs to produce a ranked cross-framework comparison with best commands per framework.

Do I need specific config files to start cross-framework LLM benchmarking?▼

Yes, you need cookbook configs and validator tools to start cross-framework LLM benchmarking. These configs drive the workflow, ensuring identical workloads are applied across SGLang, vLLM, and TensorRT-LLM to generate a normalized results report.

Related Skills

Explore More Software Engineering Skills→
hey-konghey-kong

llm-serving-auto-benchmark

Generate and validate deployment commands for comparing LLM serving frameworks.

Community
Advanced
QingHuan-6QingHuan-6

llm-serving-auto-benchmark

Benchmark SGLang, vLLM, and TensorRT-LLM serving to produce Markdown/CSV summaries.

Community
Advanced
annealing-inversionannealing-inversion

llm-serving-auto-benchmark

Benchmark LLM serving frameworks to identify the best deployment command.

Community
Advanced
dyyounggdyyoungg

sglang-sota-performance

Benchmark LLM serving frameworks and generate profiler traces for performance gap analysis.

Community
Advanced