bitsandbytes
CommunityQuantize LLMs to fit memory-limited GPUs.
AuthorCHENyiru3
Version1.0.0
Installs0
System Documentation
What problem does it solve?
BitsAndBytes enables memory-efficient quantization of large language models, allowing 8-bit and 4-bit loading and NF4 quantization to fit large models on GPUs with limited memory.
Core Features & Use Cases
- 8-bit and 4-bit quantization for efficient inference
- NF4 quantization and mixed INT8 options
- Seamless integration with transformers for loading, PEFT for fine-tuning, and DeepSpeed for distributed training
Quick Start
Install bitsandbytes and load a quantized model with 8-bit or 4-bit precision to run large models on limited GPU memory.
Dependency Matrix
Required Modules
None requiredComponents
Standard package💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: bitsandbytes Download link: https://github.com/CHENyiru3/AI-Skills-Collections/archive/main.zip#bitsandbytes Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 471,000+ vetted skills library on demand.