eval-transcription

What problem does it solve?

Transcription quality in evolving systems can drift when models or pipelines change; this skill provides formal evaluation by measuring WER, language detection accuracy, hallucination rate, and latency against a baseline, preventing regressions from being shipped.

Core Features & Use Cases

• Automated evaluation after changes to the transcription pipeline (daemon.py, model selection, post-processing, or audio input).
• Baseline governance: store baseline metrics and enforce blocking thresholds to prevent regressions.
• Multilingual evaluation and latency profiling across languages to guide model choices.

Quick Start

Run the eval after each change to the transcription pipeline to validate metrics against the baseline.