skills.rest
vivekgoquest
vivekgoquestCOMMUNITY
Explore all skills by this author →
GitHub

peft

Fine-tune large language models with LoRA and QLoRA adapters on limited GPU memory.

Updated Aug 22, 2026
Search Tags:#llm#fine-tuning#LoRA#adapters#quantization#PEFT#QLoRA
One-click install
npx skills add https://github.com/vivekgoquest/hermes-agent-stable --skill peft-vivekgoquest
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: peft
Source: https://github.com/vivekgoquest/hermes-agent-stable/tree/main/optional-skills/mlops/peft
Command: npx skills add https://github.com/vivekgoquest/hermes-agent-stable --skill peft-vivekgoquest

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires peft, transformers, torch, bitsandbytes, and includes references (resource) components.

What problem does it solve? Full fine-tuning of 7B-70B parameter LLMs requires 60+ GB of GPU memory, which is out of reach for most consumer hardware. This Skill trains less than 1% of model parameters using LoRA, QLoRA, and other adapter methods, producing 6MB adapter files instead of 14GB full model checkpoints. ## Core Features & Use Cases - LoRA and QLoRA Fine-Tuning: Train adapters on consumer GPUs, including 70B models on a single 24GB GPU via 4-bit quantization. - Multi-Adapter Serving: Load, switch, and merge multiple task-specific adapters on one base model at runtime. - Framework Integration: Works with TRL SFTTrainer, Axolotl YAML configs, and vLLM for adapter-based inference. - Use Case: Fine-tune Llama 3.1 8B on the Dolly-15k instruction dataset with rank-16 LoRA on an RTX 4090, then merge the adapter and deploy the model through vLLM. ## Quick Start Fine-tune Llama 3.1 8B with a rank-16 LoRA adapter on my instruction dataset and save only the adapter weights.

Frequently Asked Questions about peft

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with LoRA?▼

Load the base model with transformers, wrap it with get_peft_model using a LoraConfig specifying rank, alpha, and target modules, then train with the standard Trainer. Only the adapter weights are saved, typically a few megabytes instead of gigabytes.

What is the difference between LoRA and QLoRA?▼

QLoRA combines LoRA adapters with 4-bit NF4 quantization of the base model, cutting memory usage roughly threefold. Use QLoRA when memory is the primary constraint, such as fine-tuning 70B models on a single 24GB GPU, accepting a small quality trade-off.

What LoRA rank should I use for fine-tuning?▼

Start with rank 8 to 16 for general fine-tuning, which trains around 7-14M parameters. Increase to 32 or 64 for complex tasks, domain adaptation, or 70B models, and set lora_alpha to roughly twice the rank.

Can I serve multiple LoRA adapters on one base model?▼

Yes, load additional adapters with load_adapter and switch between them at runtime using set_adapter. vLLM also supports concurrent adapter serving through LoRARequest with enable_lora set to true.

Why does LoRA training run out of CUDA memory?▼

OOM occurs from large batch sizes, high rank, or missing gradient checkpointing. Enable gradient checkpointing, reduce per-device batch size with gradient accumulation, lower the rank, or switch to QLoRA with 4-bit quantization.

When should I use full fine-tuning instead of PEFT?▼

Use full fine-tuning for models under 1B parameters, when maximum quality is required and compute budget allows, or when significant domain shift requires updating all weights. PEFT suits 7B-70B models on limited hardware.

Related Skills

Explore More Software Engineering Skills→
cuba6112cuba6112

unsloth-sft

Standardize instruction and conversational datasets for SFTTrainer fine-tuning.

Community
Intermediate
ScientiaCapitalScientiaCapital
1

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

Community
Advanced
daeha76daeha76
2

evaluating-llms-harness

Evaluate LLMs against 60+ academic benchmarks using lm-evaluation-harness.

Community
Advanced
BagelHoleBagelHole
46

llm-fine-tuning

Set up QLoRA, LoRA, and full fine-tuning infrastructure with Hugging Face TRL and Axolotl.

Community
Advanced