peft-fine-tuning

Parameter-efficient fine-tuning (PEFT) reduces memory needs by training a small set of adapters, enabling 7B–70B model fine-tuning on consumer GPUs with less than 1% of total parameters.

Yes, LoRA and QLoRA are supported PEFT methods that enable training with drastically fewer trainable parameters. QLoRA additionally utilizes 4-bit quantization via bitsandbytes to further minimize resource requirements.

Multi-adapter serving is supported to enable efficient experimentation and deployment. This allows you to quickly adapt a base model to multiple domains or tasks across 7B–70B models rather than full-model finetuning.

You need the peft, transformers, and torch libraries to set up parameter-efficient fine-tuning. The bitsandbytes library is optional and required only if you want to utilize 4-bit quantization and model parallelism.

The best way is using parameter-efficient fine-tuning to train small adapters for each domain. This approach supports quick adaptation across 7B–70B models, avoiding the massive compute and memory costs of full-model finetuning.

You should avoid parameter-efficient fine-tuning if your task requires updating the base model's core weights rather than adapting behavior via adapters, as PEFT intentionally leaves the base weights frozen to save memory.