Benchmark Whisper-family ASR models on FLEURS datasets with WER and CER metrics.
npx skills add https://github.com/surus-lat/benchy --skill whisper-benchmark
Please help me install this Agent Skill. Skill: whisper-benchmark Source: https://github.com/surus-lat/benchy/tree/main/.agent/skills/whisper-benchmark Command: npx skills add https://github.com/surus-lat/benchy --skill whisper-benchmark
This skill addresses the difficulty of evaluating speech-to-text model performance locally on Apple Silicon, specifically handling hardware-specific bottlenecks and provider configurations.
Run the whisper-benchmark skill to evaluate the whisper-small model on the FLEURS Spanish dataset with a limit of three samples.
High-intent search queries and answers about installing and using this skill.
Benchmark Whisper speech-to-text models locally on Mac hardware by running transformer-based ASR models against FLEURS datasets to generate Word Error Rate and Character Error Rate metrics without requiring cloud API keys.
Yes, you can evaluate Whisper transcription accuracy on Spanish and Portuguese audio samples without cloud APIs by running local inference on FLEURS datasets. This generates comparative metrics including Word Error Rate and Character Error Rate for model variants from tiny to large-v3.
Yes, local speech-to-text benchmarking on Mac hardware requires Python 3.12 and uv, alongside dependencies like torch, transformers, librosa, and nemo-toolkit, to manage hardware-specific inference constraints on Apple Silicon.
The best way to compare Whisper model variants for local transcription throughput is evaluating accuracy and speed across FLEURS datasets using the transformers_audio provider. This generates comparative metrics to determine the optimal model variant for your Mac environment.
Benchmarking local ASR models handles hardware-specific inference constraints on Apple Silicon by applying specific model configurations via the transformers_audio provider to manage bottlenecks during speech-to-text evaluation.
guoqiaoTranscribe audio files locally using MLX on Apple Silicon.
letta-aiConfigure Nginx logging, rate limiting, and custom error pages.
robotlearning123Run Hugging Face Transformers models for NLP, vision, and audio tasks.
surus-latBenchmark local ASR models on FLEURS datasets with WER and CER tables.
