quant-recipe-search

Design and manage iterative optimization searches for quantization recipes.

3.4k|535|Updated Apr 23, 2024

Search Tags:#quantization #model optimization #ptq #llm inference #modelopt #autiquant

Ready to run

npx skills add https://github.com/NVIDIA/Model-Optimizer --skill quant-recipe-search

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: quant-recipe-search
Source: https://github.com/NVIDIA/Model-Optimizer/tree/main/.agents/skills/quant-recipe-search
Command: npx skills add https://github.com/NVIDIA/Model-Optimizer --skill quant-recipe-search

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps users find the best quantization strategy for a model by turning optimization goals, accuracy constraints, and performance targets into a structured recipe search process.

Core Features & Use Cases

Recipe Search Strategy: Designs iterative quantization experiments across formats, calibration methods, module selections, and runtime constraints.
Candidate Evaluation Planning: Establishes baselines, comparison criteria, and promotion rules for selecting validated optimization recipes.
Use Case: Optimize a large language model for inference throughput or memory reduction by exploring ModelOpt quantization candidates while preserving benchmark quality.

Quick Start

Use the quant-recipe-search skill to find the best quantization recipe for my model with a focus on throughput, memory usage, and benchmark accuracy.

Frequently Asked Questions about quant-recipe-search

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I find the best quantization recipe for large language model inference?▼

AutoQuant exploration systematically designs iterative quantization experiments across formats and calibration strategies, establishing candidate tracking and promotion rules to select validated optimization recipes that preserve benchmark quality.

How do I structure PTQ calibration strategies for model compression?▼

To structure PTQ calibration strategies, establish baselines, define comparison criteria, and set promotion rules for selecting validated optimization recipes that meet accuracy constraints and runtime compatibility checks.

What is the process for benchmark validation during quantization?▼

Benchmark validation during quantization requires tracking structured candidates, checking runtime compatibility, and comparing optimized model performance against baseline model performance to ensure accuracy constraints are met.

Can I use ModelOpt for deployment-aware quantization recipe selection?▼

Yes, you can use ModelOpt for deployment-aware quantization recipe selection by exploring candidates, validating benchmark quality, and checking runtime compatibility to optimize throughput and memory reduction.

When do I need iterative quantization searches for machine learning models?▼

You need iterative quantization searches when optimizing machine learning models for inference throughput or memory reduction, requiring structured candidate tracking and benchmark validation to preserve baseline quality.