ml-benchmark

Automate ML model benchmarking across versions and languages with JSON and Markdown reports.

12|1|Updated Nov 23, 2025

Search Tags:#evaluation #latency #benchmark #throughput #ML

Ready to run

npx skills add https://github.com/jiunbae/agent-skills --skill ml-benchmark

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: ml-benchmark
Source: https://github.com/jiunbae/agent-skills/tree/main/ml/ml-benchmark
Command: npx skills add https://github.com/jiunbae/agent-skills --skill ml-benchmark

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires python, numpy, pandas, matplotlib, and includes scripts (resource) components.

What problem does it solve?

ML 모델의 성능 벤치마크를 자동으로 실행하고, 여러 버전 간의 성능 차이를 한 눈에 비교해 의사결정을 빠르게 돕습니다.

Core Features & Use Cases

벤치마크 실행: 지연시간, 처리량, GPU 메모리 측정
결과 비교: 모델 간/버전 간 성능 비교
리포트 생성: Markdown/JSON 형식의 보고서 생성
프로파일 지원: 사전 정의된 벤치마크 시나리오 활용
다국어 평가: 언어별 성능 분석

Quick Start

예: "langdetector 벤치마크 100회 돌려줘"

Frequently Asked Questions about ml-benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I benchmark ML model latency and throughput across versions?▼

ML model benchmarking automates performance measurement across multiple versions, languages, and scenarios to capture latency (P50/P95/P99), throughput, GPU memory, and accuracy metrics. Use the provided Bash script with YAML profiles and CLI commands (run, evaluate, compare, history) to execute benchmarks and generate JSON/Markdown comparative reports.

Can I compare performance metrics between different ML models automatically?▼

Yes. The benchmarking workflow measures latency, throughput, GPU memory, and accuracy across multiple models simultaneously, then generates comparative reports in JSON and Markdown formats to highlight performance differences and support rapid decision-making.

What Python dependencies do I need to run ML model benchmarks?▼

You need Python, numpy, pandas, and matplotlib installed. These prerequisites enable the Bash script and CLI commands to execute benchmarks, process metrics data, and generate visualization outputs for performance comparison.

How do I set up predefined benchmark scenarios for consistent testing?▼

Use YAML profile configurations to define benchmark scenarios. These profiles standardize test parameters across runs, enabling you to measure latency, throughput, and GPU memory consistently and generate reproducible comparative reports.

Can I analyze model performance across multiple programming languages?▼

Yes. The benchmarking tool operates across multiple models, versions, and languages to measure performance metrics uniformly, allowing you to compare latency, throughput, and accuracy results across different language implementations.

What output formats does the benchmarking process produce?▼

The benchmarking workflow generates JSON and Markdown outputs. JSON formats enable programmatic result processing, while Markdown reports provide human-readable performance comparisons for stakeholder communication and decision-making.