skills.rest
NVIDIA-NeMo
NVIDIA-NeMoOFFICIAL
Explore all skills by this author →
WebsiteGitHub

nemotron-3-5-lightning-text2sql-lora

Runs LoRA fine-tuning of Nemotron-3.5 Lightning on BIRD Text2SQL data with Megatron-Bridge.

2.0k|411|Updated Oct 3, 2025
Search Tags:#fine-tuning#nemotron#bird#text2sql#moe#Megatron-Bridge#Lora
One-click install
npx skills add https://github.com/NVIDIA-NeMo/Nemotron --skill nemotron-3-5-lightning-text2sql-lora
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nemotron-3-5-lightning-text2sql-lora
Source: https://github.com/NVIDIA-NeMo/Nemotron/tree/main/usage-cookbook/Nemotron-3.5-Lightning/lora-text2sql/nemo-megatron-bridge
Command: npx skills add https://github.com/NVIDIA-NeMo/Nemotron --skill nemotron-3-5-lightning-text2sql-lora

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, transformers, datasets, megatron-bridge.

What problem does it solve?

Fine-tuning a 30B-A3B hybrid Mamba-Transformer MoE model involves many failure-prone steps: preparing chat-templated training data, converting checkpoints between Hugging Face and Megatron formats, sizing GPU parallelism, and merging LoRA adapters. This Skill guides a coding agent through the entire Nemotron-3.5 Lightning Text2SQL LoRA cookbook end-to-end, with per-step verification and known-pitfall handling.

Core Features & Use Cases

  • Four-step pipeline orchestration: BIRD data preparation (CPU), Hugging Face to Megatron-Bridge checkpoint conversion (CPU), packed-sequence LoRA training (GPU), and adapter merge back to a Hugging Face checkpoint.
  • Hardware-aware GPU planning: picks expert-parallel configuration from measured memory/epoch-time tables for 1-8 H100 GPUs, including single-GPU fallback with reduced MTP heads.
  • Per-step verification and troubleshooting: sanity checks for each stage plus guidance on expected friction such as slow first iterations, harmless startup log noise, and vLLM serving requirements.
  • Use Case: A user with two H100 GPUs asks an agent to fine-tune Nemotron-3.5 Lightning for Text2SQL; the agent gathers the checkpoint path and HF token, runs all four steps, and delivers a merged model that outputs bare SQL.

Quick Start

Ask the agent to run the Nemotron-3.5 Lightning Text2SQL LoRA cookbook on your machine, providing your checkpoint path, GPU count, output directory, and Hugging Face token.

Frequently Asked Questions about nemotron-3-5-lightning-text2sql-lora

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune Nemotron-3.5 Lightning with LoRA for Text2SQL?▼

Run the four-step Megatron-Bridge pipeline: prepare a BIRD training.jsonl with the model's chat template, convert the Hugging Face checkpoint to Megatron format, train a LoRA adapter on packed sequences with torchrun, then merge the adapter back into a Hugging Face checkpoint.

How many GPUs do I need to train Nemotron-3.5 Lightning?▼

One 80GB H100 works with REDUCE_MTP_HEADS=1 at about 79 GB peak memory, but two GPUs are the recommended default at roughly 51 GB each. Four or eight GPUs reduce peak memory to about 35 GB and 27 GB respectively, with proportionally faster epochs.

Can I serve the fine-tuned Nemotron checkpoint with Transformers?▼

No, transformers.generate() currently fails inside the model's bundled remote code, even on the base checkpoint. Serve the merged checkpoint with vLLM instead, which supports this architecture natively.

Why does the first training iteration take so long with no output?▼

The first iteration takes 1-2 minutes while CUDA graphs are captured and the MoE warms up; subsequent iterations take seconds. This is expected behavior, so do not cancel the job during this phase.

What should I do if training runs out of GPU memory?▼

Add a GPU or lower seq_length rather than enabling RECOMPUTE_ACTIVATIONS, which fails at iteration 2 with a Megatron gradient-buffer assertion. On a single GPU, REDUCE_MTP_HEADS=1 saves several GB by using one MTP head.

How do I know the Text2SQL fine-tune worked?▼

Serve the merged checkpoint and prompt it the way data prep formatted training examples; the fine-tuned model should return bare SQL. If it still answers conversationally with fenced SQL and prose, suspect the chat-template formatting upstream.

Related Skills

Explore More Software Engineering Skills→
Lhhiep-maxcodeLhhiep-maxcode

nemotron-3-ultra-text2sql-lora

Run the Nemotron-3 Ultra Text2SQL LoRA cookbook on a SLURM cluster.

Community
Advanced
iani-kuliiani-kuli

huggingface-community-evals

Execute local Hugging Face model evaluations with inspect-ai or lighteval frameworks.

Community
Intermediate
JeffallanJeffallan
10.9k

fine-tuning-expert

Automate fine-tuning workflows for large language models using LoRA/QLoRA and PEFT methods.

Community
Advanced
OpenLAIROpenLAIR
1.0k

peft-fine-tuning

Fine-tune 7B-70B language models with LoRA/QLoRA adapters.

Official
Advanced