skills.rest
sgl-project
sgl-projectOFFICIAL
Explore all skills by this author →
GitHub

running-eval-suite

Automate performance benchmark execution and result table synchronization in sglang-omni.

768|316|Updated Jan 7, 2026
Search Tags:#automation#performance#benchmarking#evaluation#reporting#sglang
One-click install
npx skills add https://github.com/sgl-project/sglang-omni --skill running-eval-suite
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: running-eval-suite
Source: https://github.com/sgl-project/sglang-omni/tree/main/.claude/skills/running-eval-suite
Command: npx skills add https://github.com/sgl-project/sglang-omni --skill running-eval-suite

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, sglang, torch, funasr, zhconv, zhon, and includes scripts (resource) components.

What problem does it solve?

This Skill automates the repetitive and error-prone process of running performance benchmarks and manually updating reference tables in documentation, ensuring that performance metrics remain accurate and up-to-date across different hardware configurations.

Core Features & Use Cases

  • Automated Benchmarking: Executes full-stack performance evaluations for SGLang-Omni models, including server lifecycle management and client-side data collection.
  • Intelligent Table Updates: Automatically detects host hardware and performs in-place updates or appends new rows to benchmark result tables in the repository.
  • Use Case: A developer can run a single command to evaluate a new model on H200 hardware, automatically capture the results, and commit the updated reference tables to the repository without manual editing.

Quick Start

Run the running-eval-suite skill to execute all benchmarks and refresh the reference tables for the current hardware.

Frequently Asked Questions about running-eval-suite

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate performance benchmarking for multimodal and speech models?▼

Automated performance benchmarking for multimodal and speech models is achieved by managing server lifecycles, running client execution, and parsing structured data to generate hardware-specific reporting.

What is the best way to automatically update benchmark result tables in documentation?▼

Automatically updating benchmark result tables is handled by detecting host hardware and performing in-place updates or appending new rows to reference tables in the repository without manual editing.

How does automated benchmark execution handle different hardware configurations?▼

Automated benchmark execution handles different hardware configurations by detecting the host hardware environment and producing deterministic, hardware-specific performance reporting for the evaluated models.

Do I need specific dependencies to run SGLang performance evaluations?▼

Running SGLang performance evaluations requires dependencies including pyyaml, torch, funasr, zhconv, and zhon to support multimodal and speech model processing within the sglang-omni repository.

Can I evaluate a new model on H200 hardware and commit updated reference tables automatically?▼

Evaluating a new model on H200 hardware and committing updated reference tables is supported by running a single command to capture results and synchronize documentation without manual intervention.

Related Skills

Explore More Software Engineering Skills→
Fudan-SMI-labFudan-SMI-lab
39

torch-npu-pyyaml-preinstall

Preinstall PyYAML before torch-npu to prevent transitive dependency failures.

Official
Intermediate
adriangmrraaadriangmrraa

Smart Doc Keeper

Merges new documentation into existing Markdown files while preserving original content and structure.

Community
Intermediate
achmfachmf

react-best-practices

Optimizes React and Next.js apps by fixing data fetching, bundle size, and rendering issues.

Community
Advanced
MetriLLMMetriLLM
5

metrillm

Benchmark local LLMs for performance, quality, and RAM compatibility.

Official
Intermediate