skills.rest
wshobson
wshobsonCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

lora-qlora-recipes

Configure LoRA and QLoRA adapter hyperparameters for supervised fine-tuning of language models.

39.3k|4.2k|Updated Jul 24, 2025
Search Tags:#fine-tuning#hyperparameters#peft#llm training#Lora#qLoRA#UnSloth
One-click install
npx skills add https://github.com/wshobson/agents --skill lora-qlora-recipes
Or copy as Structured Prompt for Agentâ–¼
Please help me install this Agent Skill.
Skill: lora-qlora-recipes
Source: https://github.com/wshobson/agents/tree/main/plugins/llm-finetuning/skills/lora-qlora-recipes
Command: npx skills add https://github.com/wshobson/agents --skill lora-qlora-recipes

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires unsloth, trl, peft, torch, transformers, bitsandbytes, and includes references (resource) components.

What problem does it solve?

Choosing LoRA/QLoRA hyperparameters — rank, alpha, target modules, learning rate, batch size — is error-prone, and common misconfigurations (attention-only targeting, full-FT learning rates, fp16 fallback) silently degrade training quality or cause divergence.

Core Features & Use Cases

  • Reference Recipe: Applies the settled "LoRA Without Regret" conventions — all-linear target modules, lora_alpha = 2 * r, rank by task type, and effective batch size under 32.
  • Unsloth Defaults & Mapping: Documents Unsloth's out-of-the-box defaults (lora_dropout=0, bias="none", adamw_8bit) and maps every Unsloth kwarg to its plain TRL/PEFT equivalent, including current TRL API changes like processing_class and max_length.
  • Failure Mode Diagnosis: Identifies config-level causes of loss spikes, overfitting, and OOMs — such as fp16 on non-BF16 hardware or rank too high for a small dataset — before debugging the training loop.
  • Use Case: When writing a QLoRA config for a 65B-class model on 48GB VRAM, use this Skill to produce a validated adapter config (r=32, alpha=64, 2e-4 LR, NF4 quantization) that a training script consumes directly.

Quick Start

Ask the AI to generate a LoRA or QLoRA fine-tuning configuration for your model size and task, and it will produce validated hyperparameter values following current best practices.

Frequently Asked Questions about lora-qlora-recipes

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose LoRA rank and alpha for fine-tuning?â–¼

Set lora_alpha to 2 times the rank and pick rank by task: 1–32 for RL adapters, 16–32 as a general SFT default, and up to ~256 only for large diverse datasets. Derive alpha from rank rather than tuning it independently.

What learning rate should I use for QLoRA training?â–¼

Use 2e-4 as the standard QLoRA starting point. LoRA learning rates run roughly 10x the equivalent full fine-tune LR; for conservative LoRA runs use 1e-4, or 5e-5 for continuing runs or fine-grained adjustments.

LoRA vs QLoRA vs full fine-tuning: which should I use?â–¼

Default to LoRA for adapting behavior on demonstration data. Switch to QLoRA only when the base model does not fit in bf16 at the target rank, since NF4 quantization is the memory win. Reserve full fine-tuning for injecting dense new domain knowledge.

Which target modules should LoRA adapters cover?â–¼

Target all linear modules: q_proj, k_proj, v_proj, o_proj plus the MLP layers gate_proj, up_proj, and down_proj. Attention-only targeting is the older, weaker convention, and dropping MLP modules saves negligible memory while hurting quality.

Why does QLoRA OOM on DGX Spark even though it uses less memory?â–¼

Bitsandbytes dequantization buffers are transient CUDA allocations that spike during model load, so QLoRA can OOM before an equivalent bf16 LoRA run would. Try bf16 LoRA next rather than shrinking the QLoRA config further.

When should I use plain TRL instead of Unsloth for SFT?â–¼

Use plain TRL plus PEFT for messages-shaped conversational datasets with assistant_only_loss=True, since Unsloth 2026.7.x has no messages-shaped dataset path. For other modes, fall back to TRL only after reproducing a specific Unsloth regression.

Related Skills

Explore More Software Engineering Skills→
cabeznocabezno

fine-tuning-with-trl

Fine-tune transformer language models with TRL methods for RLHF-style alignment.

Community
Advanced
KarlinskySKarlinskyS

unsloth

Fine-tune large language models with Unsloth using LoRA/QLoRA adapters.

Community
Advanced
anionzoanionzo

book-sft-pipeline

Fine-tune base models with LoRA on ePub-derived book text to emulate author voices.

Community
Advanced
cryptopaficryptopafi

grpo-rl-training

Guide GRPO/RL fine-tuning with TRL using reward-function patterns and templates.

Community
Advanced