skills.rest
yo-steven
yo-stevenCOMMUNITY
Explore all skills by this author →
GitHub

ptq

Quantize pretrained LLMs and VLMs into HuggingFace or TensorRT-LLM checkpoints.

Updated May 23, 2026
Search Tags:#calibration#nvfp4#modelopt#post-training quantization#llm quantization#hf_ptq.py#checkpoint validation
One-click install
npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill ptq-yo-steven
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ptq
Source: https://github.com/yo-steven/skills-exploration-20260522/tree/main/skills/Model-Optimizer/ptq
Command: npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill ptq-yo-steven

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill reduces the effort and risk of post-training quantization by guiding users through producing a correct quantized checkpoint (weights + config) from a pretrained model.

Core Features & Use Cases

  • End-to-end PTQ workflow: supports the full loop from environment setup, to choosing a quantization format, to exporting a calibrated checkpoint.
  • Model support handling: uses the verified support matrix when possible, and provides a structured path for unlisted models with smoke-test-first iteration.
  • Guardrails for correctness: includes explicit validation guidance to detect silently-missed layers via checkpoint validation checks after export.

Quick Start

Use the ptq skill to quantize Qwen3-0.6B to nvfp4 and export an INT8/INT4-style calibrated checkpoint directory for later deployment.

Frequently Asked Questions about ptq

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run post-training quantization on a pretrained LLM using NVIDIA ModelOpt?▼

Run post-training quantization on a pretrained LLM using NVIDIA ModelOpt by executing the hf_ptq.py script with the correct qformat and calibration settings to export a calibrated checkpoint directory for deployment.

What quantization formats are supported for LLM quantization with ModelOpt?▼

ModelOpt LLM quantization supports formats including NVFP4, FP8, INT8, and INT4_AWQ, allowing you to apply the correct qformat setting in the hf_ptq.py script to produce the desired calibrated checkpoint.

How do I validate a quantized checkpoint to ensure no layers were silently missed?▼

Validate a quantized checkpoint by performing explicit checkpoint validation checks after export to detect silently missed layers and verify the applied quantization pattern matches the expected configuration.

Can I quantize an unlisted model that is not in the LLM PTQ support matrix?▼

You can quantize an unlisted model by following a structured path that prioritizes smoke-test-first iteration, bypassing the verified support matrix to safely handle unverified models during the post-training quantization workflow.

Do I need to configure trust_remote_code when performing PTQ on HuggingFace checkpoints?▼

You need to handle trust_remote_code dependencies when performing PTQ on HuggingFace checkpoints to ensure the correct execution of custom model code during the calibration and quantization process.

Related Skills

Explore More Software Engineering Skills→
NVIDIANVIDIA
2.8k

ptq

Quantize pretrained models to nvfp4, fp8, or int4_awq checkpoints using ModelOpt PTQ.

Official
Advanced
jayll1303jayll1303
17

tensorrt-llm

Convert HuggingFace checkpoints into optimized TensorRT-LLM engines with quantization.

Community
Advanced
AlvaroBianoAlvaroBiano

tensorrt-llm

Accelerate LLM inference on NVIDIA GPUs with TensorRT-LLM optimizations.

Community
Advanced
Orchestra-ResearchOrchestra-Research
11.5k

hqq-quantization

Quantize LLM weights to 8/4/3/2/1-bit precision without calibration.

Official
Intermediate