Runs ModelOpt Quantization-Aware Distillation on Slurm through Megatron Bridge to recover PTQ accuracy gaps.
npx skills add https://github.com/NVIDIA/Model-Optimizer --skill qad
Please help me install this Agent Skill. Skill: qad Source: https://github.com/NVIDIA/Model-Optimizer/tree/main/plugins/modelopt/skills/qad Command: npx skills add https://github.com/NVIDIA/Model-Optimizer --skill qad
Post-training quantization (PTQ) can introduce accuracy regressions relative to BF16 baselines, and recovering that gap requires expensive, carefully orchestrated Quantization-Aware Distillation (QAD) training on multi-node Slurm clusters. This Skill governs the full QAD workflow so it is only launched when explicitly authorized and executed with validated baselines, correct topology, and disciplined stop conditions.
Run QAD with ModelOpt on my Slurm cluster to recover the accuracy gap between my BF16 baseline and the PTQ checkpoint of my model.
High-intent search queries and answers about installing and using this skill.
Run QAD through the Megatron Bridge examples using quantize.py for PTQ and distill.py for distillation training on Slurm. First confirm a measured BF16-to-PTQ accuracy gap of at least 1%, then prepare the dataset, choose topology, and submit sequential dependent training jobs.
Use QAD only when PTQ leaves an accuracy gap of at least 1% relative to BF16 on your target benchmarks and you explicitly authorize the expensive training run. If the PTQ gap is already below 1%, QAD is not justified.
Yes, but with constraints: DP and EDP must be integral, num_experts must be divisible by EP, and GBS must be divisible by MBS times DP. Expert tensor parallelism (ETP) must stay at 1 because the current distill.py workflow does not support it.
Resume from the latest QAD checkpoint in the same output directory with unchanged prepared data paths, seed, topology, optimizer, scheduler, iteration, and consumed-sample state. Do not restart from the PTQ checkpoint.
Defaults include 32768 sequence length, 1e-5 peak and 1e-6 minimum learning rate with cosine schedule, 1000 training iterations, global batch size 512, checkpoints every 50 iterations, and recovery benchmarks at iteration 150 then every 100.
Cancel on non-finite loss, repeated skipped iterations, or a sustained loss spike. Also cancel when the full-suite gap to BF16 drops below 1%, benchmark recovery regresses beyond run noise, or both recovery and loss plateau.
QingHuan-6Quantize diffusion DiT models with ModelOpt to FP8 or NVFP4 checkpoints for SGLang Diffusion.
QingHuan-6Benchmark SGLang, vLLM, and TensorRT-LLM serving to produce Markdown/CSV summaries.
NVIDIA-TAOAutomate remote TAO GPU job submission on SLURM clusters via SSH.
yo-stevenQuantize pretrained LLMs and VLMs into HuggingFace or TensorRT-LLM checkpoints.
