skills.rest
amd
amdOFFICIAL
Explore all skills by this author →
WebsiteGitHub

quark-onnx-ptq-workflow

Guide end-to-end post-training quantization of ONNX models with AMD Quark.

154|27|Updated Jun 19, 2024
Search Tags:#calibration#quantization#model optimization#onnx#post-training quantization#amd quark
One-click install
npx skills add https://github.com/amd/Quark --skill quark-onnx-ptq-workflow
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: quark-onnx-ptq-workflow
Source: https://github.com/amd/Quark/tree/main/.claude/skills-impl/l2-workflows/onnx/quark-onnx-ptq-workflow
Command: npx skills add https://github.com/amd/Quark --skill quark-onnx-ptq-workflow

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill removes the manual friction from end-to-end post-training quantization of ONNX models with AMD Quark, guiding the user from model intake through planning, script generation, confirmation, and execution.

Core Features & Use Cases

  • Model intake and validation: Analyzes an input .onnx model for opset, shapes, quantizable operators, already-quantized QDQ nodes, and deployment compatibility risks.
  • Quantization planning: Builds a concrete PTQ plan for presets such as XINT8, A8W8, A16W8, BF16, BFP16, and MX/MXFP variants, including calibration settings and algorithm choices.
  • Manifest and script generation: Produces a standalone calibration and quantization script plus a run manifest, with guardrails that avoid modifying the Quark repository and preserve reproducibility.
  • Safe execution workflow: Requires checkpointed user confirmation before running quantization and surfaces failures for debugging instead of silently altering providers or patching upstream code.
  • Use case: Quantize a vision or LLM ONNX model for CPU, CUDA, ROCm, or AMD NPU deployment while keeping calibration data, output artifacts, and configuration decisions explicit and auditable.

Quick Start

Use the quark-onnx-ptq-workflow skill to analyze my ONNX model, propose a quantization plan, generate the runnable script and manifest, and wait for my confirmation before execution.

Frequently Asked Questions about quark-onnx-ptq-workflow

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I quantize an ONNX model for AMD NPU deployment?▼

To quantize an ONNX model for AMD NPU deployment, you need a structured post-training quantization workflow that analyzes model operators, plans calibration, generates a standalone script, and executes with user confirmation. This supports both CNN and Transformer NPU targets.

What is the best way to perform post-training quantization on ONNX models?▼

Post-training quantization on ONNX models works best using an end-to-end workflow that validates opset compatibility, selects presets like XINT8 or BF16, and generates a reproducible calibration script with an execution manifest to ensure auditable artifacts.

Can I use AMD Quark to quantize vision and language ONNX models for CUDA or ROCm?▼

Yes, AMD Quark supports post-training quantization workflows for both vision and language ONNX models targeting CPU, CUDA, and ROCm deployments, providing explicit calibration data management and configuration decisions for each target environment.

How does calibration work during ONNX model quantization?▼

Calibration during ONNX model quantization involves analyzing model shapes and quantizable operators, applying algorithm choices for presets like A8W8 or MXFP, and generating a standalone script that processes calibration data before producing the final quantized model artifacts.

What are the limitations of automated ONNX model quantization workflows?▼

Automated ONNX model quantization workflows require checkpointed user confirmation before execution to prevent silent failures, and they avoid modifying the upstream Quark repository to preserve reproducibility, meaning deployment compatibility risks must be resolved during initial model intake.

Do I need to inspect my ONNX model before starting quantization?▼

Yes, inspecting your ONNX model before quantization is required to evaluate the opset, shapes, quantizable operators, and existing QDQ nodes, ensuring the model meets deployment compatibility requirements and allowing the workflow to build an accurate quantization plan.

Related Skills

Explore More Software Engineering Skills→
amdamd
154

quark-onnx-model-intake

Extracts ONNX model metadata needed for Quark PTQ planning and compatibility assessment.

Official
Advanced
amdamd
154

quark-onnx-quant-plan

Build hardware-aware Quark ONNX quantization plans from model analysis and deployment intent.

Official
Advanced
ihatesea69ihatesea69
3

llama-cpp

Run GGUF quantized LLM inference on CPU, Apple Silicon, and consumer GPUs.

Community
Advanced
ovachieverovachiever
52

quantizing-models-bitsandbytes

Quantize LLMs to 8-bit or 4-bit precision using BitsAndBytes.

Community
Advanced