skills.rest
MLT-OSS
MLT-OSSOFFICIAL
Explore all skills by this author →
WebsiteGitHub

peft-fine-tuning

Fine-tune LLMs with LoRA, QLoRA, and adapter methods using HuggingFace PEFT.

14|5|Updated Apr 9, 2026
Search Tags:#llm#fine-tuning#adapters#lora#huggingface#peft#qlora
One-click install
npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill peft-fine-tuning-mlt-oss
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: peft-fine-tuning
Source: https://github.com/MLT-OSS/hermes-agent-go/tree/main/skills/mlops/training/peft
Command: npx skills add https://github.com/MLT-OSS/hermes-agent-go --skill peft-fine-tuning-mlt-oss

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires peft, transformers, torch, bitsandbytes, and includes references (resource) components.

What problem does it solve? Full fine-tuning of large language models (7B-70B) requires enormous GPU memory and produces multi-gigabyte checkpoints, making it impractical on consumer hardware. This Skill enables parameter-efficient fine-tuning that trains under 1% of model parameters, producing megabyte-sized adapters instead of full model copies. ## Core Features & Use Cases - LoRA and QLoRA Fine-Tuning: Train low-rank adapters on models from 7B to 70B parameters, with 4-bit quantization support to fit a 70B model on a single 24GB GPU. - 25+ PEFT Methods: Includes AdaLoRA, IA3, Prefix Tuning, Prompt Tuning, DoRA, LoftQ, and rank-stabilized LoRA with guidance on when each method applies. - Multi-Adapter Serving: Load, switch, merge, and compose multiple adapters on one base model for deployment with vLLM or HuggingFace Transformers. - Use Case: Fine-tune Llama 3.1 8B on the Dolly-15k instruction dataset with rank-16 LoRA on a single RTX 4090, producing a 6MB adapter that merges into the base model for deployment. ## Quick Start Fine-tune Llama 3.1 8B with LoRA rank 16 on my instruction dataset and save the adapter to ./lora-adapter.

Frequently Asked Questions about peft-fine-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with LoRA?▼

Load the base model with AutoModelForCausalLM, create a LoraConfig with rank 8-16 and target modules like q_proj and v_proj, then wrap the model with get_peft_model. Train with the standard HuggingFace Trainer and save only the adapter weights with save_pretrained.

What is the difference between LoRA and QLoRA?▼

QLoRA combines LoRA adapters with 4-bit NF4 quantization of the base model via bitsandbytes, cutting memory from around 18GB to 6GB for an 8B model. Use QLoRA when memory is the primary constraint, accepting roughly a 5% quality trade-off versus standard LoRA.

What LoRA rank should I use for fine-tuning?▼

Start with rank 8-16 for general fine-tuning, which trains about 0.17% of parameters on an 8B model. Use rank 32-64 for complex tasks, domain adaptation, or 70B models, and set lora_alpha to twice the rank as a starting point.

Can I serve multiple LoRA adapters on one base model?▼

Yes, PEFT supports loading multiple adapters with load_adapter and switching between them at runtime using set_adapter. For production serving, vLLM supports concurrent LoRA requests with enable_lora=True and configurable max_loras.

Why am I getting CUDA out of memory during LoRA training?▼

Enable gradient checkpointing with prepare_model_for_kbit_training, reduce per-device batch size while increasing gradient accumulation steps, or switch to QLoRA with 4-bit quantization. Lowering the LoRA rank and targeting fewer modules also reduces memory usage.

When should I use full fine-tuning instead of PEFT?▼

Use full fine-tuning for models under 1B parameters, when maximum quality is required and compute budget allows, or when significant domain shift requires updating all weights. PEFT methods train under 1% of parameters and may underperform in these cases.

Related Skills

Explore More Data & Analytics Skills→
zerwizzerwiz

axolotl

Configure and troubleshoot Axolotl YAML training workflows for LLM fine-tuning.

Community
Advanced
ScientiaCapitalScientiaCapital
1

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

Community
Advanced
JeffallanJeffallan
10.9k

fine-tuning-expert

Automate fine-tuning workflows for large language models using LoRA/QLoRA and PEFT methods.

Community
Advanced
NzettodessNzettodess

llama-factory

Guides fine-tuning of large language models using LLaMA-Factory with LoRA, QLoRA, and WebUI workflows.

Community
Intermediate