skills.rest
ehadziabdic
ehadziabdicCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llm-fine-tuning

Fine-tune large language models with LoRA, QLoRA, and DPO using HuggingFace PEFT and TRL.

Updated Sep 23, 2026
Search Tags:#fine-tuning#lora#huggingface#peft#qlora#dpo#llm training
One-click install
npx skills add https://github.com/ehadziabdic/WAgents --skill llm-fine-tuning-ehadziabdic
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llm-fine-tuning
Source: https://github.com/ehadziabdic/WAgents/tree/main/opencode/skills/llm-fine-tuning
Command: npx skills add https://github.com/ehadziabdic/WAgents --skill llm-fine-tuning-ehadziabdic

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, transformers, peft, trl, datasets, bitsandbytes, accelerate, liger-kernel, pandas, numpy, and includes scripts (resource) and references (resource) components.

What problem does it solve? Adapting a pre-trained LLM to a specific domain or task requires navigating method selection, dataset formatting, GPU memory constraints, and alignment training, where missteps waste hours of compute. This Skill provides decision trees, working code, and validated troubleshooting guidance for the full fine-tuning lifecycle. ## Core Features & Use Cases - Parameter-Efficient Fine-Tuning: Implements LoRA and QLoRA pipelines with HuggingFace PEFT and TRL, enabling 7B model tuning on consumer GPUs with as little as 6GB VRAM. - Dataset Preparation and Validation: Converts between Alpaca, ShareGPT, and chat formats, deduplicates records, analyzes token lengths, and splits train/validation sets via the prepare_dataset.py script. - Preference Alignment: Supports DPO training with preference pairs as a simpler alternative to full RLHF pipelines. - Use Case: Fine-tune Qwen3-8B on 5,000 domain-specific instruction examples using QLoRA on a single RTX 4090, then merge the adapter and convert to GGUF for llama.cpp deployment. ## Quick Start Fine-tune Qwen/Qwen3-8B on my train.jsonl dataset using QLoRA with rank 16 and save the adapter to ./qlora-output.

Frequently Asked Questions about llm-fine-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune an LLM with LoRA in Python?▼

Load the base model with transformers, apply a LoraConfig via peft's get_peft_model with rank 16 and alpha 32, then train with TRL's SFTTrainer. The finetune_lora.py script automates this pipeline from dataset loading through adapter saving.

LoRA vs QLoRA: which should I use for fine-tuning?▼

QLoRA quantizes the base model to 4-bit, fitting a 7B model in roughly 6-10GB VRAM, while LoRA needs 18-24GB for the same size. Start with QLoRA to test hypotheses cheaply, then scale to LoRA if quality loss from quantization matters.

What dataset format does HuggingFace SFTTrainer expect?▼

SFTTrainer accepts chat-formatted data where the tokenizer's apply_chat_template converts message arrays into training text. The prepare_dataset.py script converts Alpaca, ShareGPT, and completion formats into chat format with validation and deduplication.

Why does fine-tuning OOM at step 0 even with 4-bit quantization?▼

The fp32 cross-entropy logits tensor scales with vocab size times sequence length, reaching about 8GB for a 151,936-token vocabulary at 14,336 tokens. Setting use_liger_kernel=True in SFTConfig fuses the linear cross-entropy and avoids materializing the full logits tensor.

Should I use DPO or RLHF for model alignment?▼

DPO is preferred for most teams because it needs only preference pairs and two models, while RLHF requires a reward model plus PPO with four models and is prone to reward hacking. Always run SFT first, then apply DPO on the SFT checkpoint.

How do I deploy a fine-tuned LoRA adapter?▼

Load the base model plus adapter with PeftModel, call merge_and_unload to fold weights into the base model, then save the merged result. Optionally convert to GGUF with convert_hf_to_gguf.py and quantize with llama-quantize for llama.cpp serving.

Related Skills

Explore More Software Engineering Skills→
RachasumanthRachasumanth

data-annotation

Create, format, and validate instruction-tuning and preference datasets for AI model fine-tuning.

Community
Advanced
rabbicserabbicse

fine-tuning-expert

Optimizes LLM fine-tuning workflows with LoRA, QLora, and PEFT.

Community
Advanced
K-Dense-AIK-Dense-AI
41.1k

pytorch-lightning

Organize PyTorch code into LightningModules and configure distributed multi-GPU training workflows.

Official
Advanced
spencerpaulyspencerpauly
672

prompt-engineering

Improve LLM prompts with clear constraints, examples, and output formats.

Community
Intermediate