skills.rest
NVIDIA
NVIDIAOFFICIAL
Explore all skills by this author →
WebsiteGitHub

nemotron-asr-finetune

Orchestrates NVIDIA NeMo and Riva ASR customization workflows from word boosting to fine-tuning.

3.2k|370|Updated Feb 25, 2026
Search Tags:#fine-tuning#speech-to-text# orchestration#asr#nemo#wer#riva
One-click install
npx skills add https://github.com/NVIDIA/skills --skill nemotron-asr-finetune
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nemotron-asr-finetune
Source: https://github.com/NVIDIA/skills/tree/main/skills/nemotron-asr-finetune
Command: npx skills add https://github.com/NVIDIA/skills --skill nemotron-asr-finetune

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

Customizing NVIDIA Nemotron Speech (Riva) / NeMo ASR models for a specific domain or language involves many techniques—word boosting, custom vocabularies, n-gram language models, and full fine-tuning—and choosing the wrong one wastes GPU time and budget. This Skill scopes the problem, selects the cheapest sufficient path, and delegates each stage to the correct sub-skill.

Core Features & Use Cases

  • Guided Path Selection: Diagnoses the failure mode (jargon, phrasing, acoustics, new language) and picks the lowest-cost rung, escalating only when quality targets are missed.
  • Sub-Skill Orchestration: Routes data generation, NeMo training, WER evaluation, and Riva NIM deployment to the right sub-skills with defined handoff contracts.
  • Planning Answers: Provides data-volume, synthetic-vs-real, GPU-choice (L40S vs H100), and cost guidance with stated assumptions.
  • Use Case: A team whose Riva ASR mishears noisy call-center audio gets a scoped plan: confirm 100+ hours of transcribed audio, fine-tune via NeMo, evaluate normalized WER plus a forgetting check, then deploy through a Riva NIM.

Quick Start

Ask your agent to help you improve ASR accuracy for your domain, for example: "My Riva ASR model gets our German medical jargon wrong—what is the cheapest way to fix it?"

Frequently Asked Questions about nemotron-asr-finetune

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune NVIDIA Riva ASR for my domain?▼

Start with scoping: measure a baseline WER, inventory real transcribed audio, and define a target. The skill then picks the cheapest sufficient rung—word boosting, n-gram LM, or fine-tuning—and delegates training to the nemo-speech-asr-finetune sub-skill and deployment to nemotron-speech.

What is the cheapest way to improve ASR accuracy on jargon?▼

Word boosting is the cheapest fix for a bounded set of known words, names, or jargon—it runs at serving time with no training. For out-of-vocabulary or mispronounced terms, use custom vocabulary and pronunciation at deploy time before considering an n-gram LM or fine-tuning.

How much data do I need to fine-tune a NeMo ASR model?▼

The NVIDIA NIM guide recommends 100+ hours of transcribed target-domain audio for fine-tuning. Around 10 hours is the floor, and only when mixed with a larger dataset to avoid catastrophic forgetting. N-gram LM rungs need only domain text, not audio.

When should I use word boosting vs fine-tuning for ASR?▼

Use word boosting when errors involve a known set of words or names; it is runtime-only and free of training. Fine-tune only for genuine acoustic gaps like accents, noise, or channel mismatch, after cheaper rungs provably miss the WER target.

Does this skill work with OpenAI Whisper or text LLMs?▼

No. The skill stays silent for OpenAI Whisper fine-tuning and text-LLM tasks such as Llama fine-tuning. It only orchestrates NVIDIA NeMo/Riva ASR customization and defers pure model export and deployment to the nemotron-speech skill.

Should I use L40S or H100 GPUs for ASR fine-tuning?▼

H100 suits large models, large batches, and fastest turnaround for full fine-tunes. L40S is more cost-effective for smaller models, evaluation, and parallel smaller jobs. Decide based on model size, precision, VRAM headroom from an OOM preflight, and cost targets.

Related Skills

Explore More Software Engineering Skills→
MS33834MS33834

RAG Retrieval Eval

Creates comprehensive evaluation plans for document retrieval systems, including metrics, benchmarks, and analysis.

Community
Intermediate
truefoundrytruefoundry
13

llm-deploy

Deploy LLMs and ML inference servers on TrueFoundry with GPU-accelerated endpoints.

Official
Advanced
automateyournetworkautomateyournetwork
627

slack-voice-interface

Transcribe Slack voice clips and generate MP3 audio replies.

Community
Intermediate
hochoa13hochoa13

whisper

Transcribe audio from 99 languages and translate it to English.

Community
Intermediate