benchmark

Benchmark on-device ASR, TTS, VAD, and diarization latency and efficiency.

1.1k|148|Updated Feb 4, 2026
One-click install
npx skills add https://github.com/soniqo/speech-swift --skill benchmark-soniqo
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: benchmark
Source: https://github.com/soniqo/speech-swift/tree/main/.claude/skills/benchmark
Command: npx skills add https://github.com/soniqo/speech-swift --skill benchmark-soniqo

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires python3, and includes scripts (resource) components.

What problem does it solve?

Benchmarking on-device ASR, TTS, VAD, and diarization performance can be hard to compare across models, builds, and hardware, making it difficult to validate real-world latency and accuracy.

Core Features & Use Cases

  • Module-scoped benchmarks: Run separate performance tests for ASR, TTS, VAD, or diarization without changing your workflow.
  • Release-build execution: Uses the release build binary for consistent performance measurement.
  • Example use case: Validate whether a newer ASR or TTS model meets expected real-time factor targets before adopting it for dictation, streaming playback, or evaluation.

Quick Start

Run the ASR benchmark on the provided test audio and collect the resulting RTF by invoking the benchmark module as asr.

Frequently Asked Questions about benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark speech recognition latency on Apple Silicon?

Benchmark speech recognition latency on Apple Silicon by executing provided Python scripts against a release build binary to quantify real-time factor and processing efficiency for ASR tasks.

Can I measure text to speech and voice activity detection performance separately?

Yes, you can measure text to speech and voice activity detection performance separately by invoking module-scoped benchmark arguments to run isolated performance tests for TTS and VAD without altering your workflow.

What test audio is needed to benchmark diarization accuracy?

To benchmark diarization accuracy, you need the provided test audio files and VoxConverse test sets to ensure repeatable comparison of real-world latency and efficiency across different builds.

Do I need a release build to measure on-device speech processing performance?

Yes, a release build binary is required to measure on-device speech processing performance because it ensures consistent performance measurement without the overhead of debug instrumentation.

How do I validate real-time factor targets before adopting a new ASR model?

Validate real-time factor targets by running the ASR benchmark module on provided test audio to collect resulting RTF metrics and compare them against your expected performance thresholds.