colab-quantization
CommunityQuantize Colab models for GPU inference with various methods
Authorkngender5
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill solves the problem of compressing machine learning models for inference on Google Colab, balancing between model size, speed, and quality.
Core Features & Use Cases
- Model Quantization: Provides a range of quantization methods to compress models (AWQ, GPTQ, GGUF, bitsandbytes 4/8-bit, HQQ, EETQ).
- Benchmarking: Offers quality vs speed vs VRAM tradeoff analysis.
- Flexibility: Allows selection of the quantization method based on the scenario (e.g., T4 inference, CPU only, no calibration, etc.).
Quick Start
To compress a Colab model with bitsandbytes, use the following command: bitsandbytes quantize "model_path" --output_path "model_bnb".
Dependency Matrix
Required Modules
transformersbitsandbytesawqgptqggufonnxruntime-gpu
Components
scriptsreferencesassets
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: colab-quantization Download link: https://github.com/kngender5/hermes/archive/main.zip#colab-quantization Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.