skills.rest
Arm-Examples
Arm-ExamplesOFFICIAL
Explore all skills by this author →
GitHub

llm-benchmark-workflow

Build and execute LLM benchmarking tools across multiple backends.

4|3|Updated Feb 18, 2026
Search Tags:#workflow#testing#performance#llm#build#benchmark
One-click install
npx skills add https://github.com/Arm-Examples/LLM-Runner --skill llm-benchmark-workflow
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-benchmark-workflow
Source: https://github.com/Arm-Examples/LLM-Runner/tree/main/skills/llm-benchmark-workflow
Command: npx skills add https://github.com/Arm-Examples/LLM-Runner --skill llm-benchmark-workflow

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of building and executing performance benchmarks for Large Language Models (LLMs) across various backends, helping to identify and resolve issues related to benchmark execution and performance.

Core Features & Use Cases

  • Benchmark Compilation: Compiles LLM benchmarking tools, including arm-llm-bench-cli, for different ML frameworks.
  • Backend Configuration: Allows selection of specific backends like llama.cpp, ONNX Runtime GenAI, MediaPipe, and MNN.
  • Runtime Triage: Assists in diagnosing and fixing build or runtime problems such as shared library placement, model path configuration, and thread/token settings.
  • Use Case: When you modify the benchmarking code or add new performance metrics, use this Skill to ensure the benchmarks build correctly and run as expected across different LLM backends.

Quick Start

Build the benchmark binaries for the native platform by running cmake --preset=native -B build -DBUILD_BENCHMARK=ON && cmake --build ./build --parallel.

Frequently Asked Questions about llm-benchmark-workflow

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build and run LLM benchmarks for different ML frameworks?▼

To build LLM benchmarks, compile tools like arm-llm-bench-cli using CMake with the BUILD_BENCHMARK flag. You can then execute these binaries across multiple supported backends to measure and compare Large Language Model performance.

Does LLM benchmarking work with ONNX Runtime GenAI and llama.cpp backends?▼

Yes, LLM benchmarking supports various backends including llama.cpp, ONNX Runtime GenAI, MediaPipe, and MNN. You can select a specific backend during configuration to evaluate LLM performance across different frameworks.

Why does my LLM benchmark fail at runtime with shared library and model path issues?▼

Runtime failures often stem from incorrect shared library placement or model path configurations. This Skill assists in triaging and fixing these build and runtime problems, including JNI settings and thread configurations.

What is the best way to verify benchmark binaries after modifying performance metrics?▼

The best way to verify benchmark binaries is to rebuild them using CMake and run the updated benchmarks across your target LLM backends. This ensures code modifications and new performance metrics execute as expected.

Can I diagnose build configuration problems for LLM benchmarking tools?▼

Yes, you can diagnose build configuration problems for LLM benchmarking tools. The Skill addresses issues related to benchmark build configurations, ensuring correct compilation and execution across different ML frameworks.

What steps are needed to compile arm-llm-bench-cli for native platforms?▼

To compile arm-llm-bench-cli for native platforms, run the CMake preset command for native builds with the BUILD_BENCHMARK option enabled, followed by the CMake build command to generate the benchmark binaries in parallel.

Related Skills

Explore More Software Engineering Skills→
steven715steven715
2

backend-patterns

Generate C++ backend project templates with CMake and vcpkg configuration.

Community
Advanced
jleechanorgjleechanorg

evaluating-llms-harness

Benchmark LLMs across 60+ tasks with standardized prompts and metrics.

Official
Advanced
nist0nist0

bruno

Execute API test collections with environment configurations in CI/CD pipelines.

Community
Advanced
gofhirgofhir
2

go-benchmarking

Benchmark and profile Go code using the standard testing framework and pprof.

Official
Intermediate