skills.rest
atrawog
atrawogCOMMUNITY
Explore all skills by this author →
Website𝕏@atrawogGitHub

quantization

Quantize neural network models to FP32, FP16, BF16, INT8, or INT4 precisions.

Updated Aug 23, 2026
Search Tags:#memory#model#quantization#inference#bitsandbytes#nf4#4bit
One-click install
npx skills add https://github.com/atrawog/overthink-plugins --skill quantization
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: quantization
Source: https://github.com/atrawog/overthink-plugins/tree/main/overthink-jupyter/skills/quantization
Command: npx skills add https://github.com/atrawog/overthink-plugins --skill quantization

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Quantizes neural network models to reduce memory footprint and accelerate inference.

Core Features & Use Cases

  • Supports FP32, FP16, BF16, INT8, INT4 precisions to balance accuracy and performance.
  • Provides BitsAndBytes-based loading configurations (load_in_4bit, nf4, fp4) and memory estimation.
  • Enables deployment on memory-constrained hardware and training with QLoRA workflows.

Quick Start

Quantize your model to 4-bit NF4 for reduced memory and faster inference.

Frequently Asked Questions about quantization

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I quantize a transformer model to 4-bit using nf4?▼

This Skill quantizes neural network models to 4-bit NF4 using BitsAndBytesConfig, reducing memory footprint and accelerating inference in memory-constrained environments while maintaining performance.

What precisions are supported for model quantization?▼

Supported quantization precisions include FP32, FP16, BF16, INT8, and INT4, allowing you to balance accuracy and performance when quantizing transformer or computer vision models.

Can I use 4-bit quantization for QLoRA training?▼

Yes, 4-bit quantization supports QLoRA training workflows by loading models with nf4 or fp4 BitsAndBytes configurations, enabling training with quantized weights on memory-constrained hardware.

Does BitsAndBytesConfig work with computer vision networks?▼

Yes, BitsAndBytesConfig quantizes both transformer-based models and computer vision networks, reducing memory footprint and accelerating inference across these neural network architectures.

What are the tradeoffs between nf4 and fp4 quantization?▼

The Skill provides guidance on performance tradeoffs between nf4 and fp4 4-bit options, balancing memory footprint reduction against accuracy retention and inference acceleration for different model architectures.

Related Skills

Explore More Software Engineering Skills→
mattmirelesmattmireles
57

coreml-validate

Validate CoreML model numerical accuracy against PyTorch reference models.

Community
Advanced
ihatesea69ihatesea69
3

quantizing-models-bitsandbytes

Quantize LLMs to 8-bit or 4-bit precision using bitsandbytes formats.

Community
Advanced
GitnappGitnapp

peft-fine-tuning

Configure LoRA and QLoRA adapters for memory-efficient LLM fine-tuning.

Community
Advanced
KappTech88KappTech88

quantizing-models-bitsandbytes

Quantize large language models to 4-bit NF4 and 8-bit configurations.

Community
Advanced