skills.rest
sayalinvidia
sayalinvidiaCOMMUNITY
Explore all skills by this author →
GitHub

digital-health-clinical-asr-eval

Score NeMo-format clinical ASR manifests and generate a five-section KER leaderboard.

Updated Apr 16, 2026
Search Tags:#routing#evaluation#manifest#scoring#leaderboard#clinical-asr#KER
One-click install
npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill digital-health-clinical-asr-eval
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: digital-health-clinical-asr-eval
Source: https://github.com/sayalinvidia/sayali-skills-test/tree/main/skills/digital-health-clinical-asr-eval
Command: npx skills add https://github.com/sayalinvidia/sayali-skills-test --skill digital-health-clinical-asr-eval

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill scores NeMo-format clinical ASR manifests, generates a five-section leaderboard, and provides routing guidance to decide subsequent steps (build, finetune, or stop) based on evaluation results.

Core Features & Use Cases

  • Leaderboard generation: Produces a fixed five-section Markdown leaderboard (Headline, KER by entity_category, KER by ipa_source, KER by noise_level, Per-term KER).
  • Inline scoring: Computes WER, CER, KER (strict contiguous term matching), and SER using a pure-Python pipeline with inlined normalization and distance metrics.
  • Post-eval routing: Applies the clinical decision tree to suggest next actions after evaluation.
  • Reference artifacts: Outputs per-sample results and a per-skill eval report, aligned with NVSkills-Eval workflows.

Quick Start

Provide a manifest.jsonl to the skill to generate the leaderboard and scoring report.

Frequently Asked Questions about digital-health-clinical-asr-eval

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I score clinical ASR manifests and generate a WER CER KER leaderboard?▼

Scoring a clinical ASR manifest requires a NeMo-format manifest.jsonl file with clinical-extension fields and an NVIDIA_API_KEY. The pipeline computes WER, CER, KER, and SER metrics to produce a five-section Markdown leaderboard with per-sample results.

What metrics are used for clinical ASR evaluation beyond standard word error rate?▼

Clinical ASR evaluation uses KER (Keyword Error Rate) alongside WER, CER, and SER. KER applies strict contiguous term matching to evaluate medical terminology recognition accuracy within the inline pure-Python scoring pipeline.

How does post-evaluation routing guide clinical ASR deployment decisions?▼

Post-evaluation routing applies a clinical decision tree to the generated leaderboard results. It suggests whether to build, finetune, or stop based on surfaced WER, CER, KER, and SER metrics across noise levels and IPA sources.

Do I need an NVIDIA API key to run clinical ASR manifest scoring?▼

Yes, an NVIDIA_API_KEY is required to run clinical ASR manifest scoring. The inline evaluation pipeline requires this key alongside a NeMo manifest with clinical-extension fields to generate the leaderboard and routing report.

Can I analyze clinical ASR performance by noise level and IPA source?▼

Yes, the generated leaderboard breaks down KER metrics by ipa_source and noise_level. It also provides per-term KER analysis and KER by entity_category to identify specific clinical terminology recognition weaknesses.

Related Skills

Explore More Software Engineering Skills→
sayalinvidiasayalinvidia

digital-health-clinical-asr-build

Curates clinical terms, annotates IPA pronunciations, and generates NeMo manifests with evaluation audio.

Community
Advanced
rongarederongarede
4

eval-harness

Evaluate Claude Code sessions with capability and regression evals.

Community
Advanced
sayalinvidiasayalinvidia

digital-health-clinical-asr-finetune

Fine-tune clinical ASR models with NeMo SFT on Parakeet TDT 0.6B.

Community
Advanced
euroconiceuroconic
1

eval-transcription

Automates transcription quality evaluation by measuring WER, language accuracy, hallucination rate, latency against a baseline.

Community
Advanced