skills.rest
Chensihakniroth
ChensihaknirothCOMMUNITY
Explore all skills by this author →
GitHub

unsloth

Fine-tune LLMs with LoRA and QLoRA using Unsloth for faster training and lower VRAM usage.

Updated Jun 7, 2026
Search Tags:#fine-tuning#LLM training#GGUF#LoRA#GRPO#QLoRA#Unsloth
One-click install
npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill unsloth-chensihakniroth
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: unsloth
Source: https://github.com/Chensihakniroth/ANAKOT-AGENT/tree/main/optional-skills/mlops/training/unsloth
Command: npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill unsloth-chensihakniroth

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires unsloth, torch, transformers, trl, datasets, peft, and includes references (resource) components.

What problem does it solve? Fine-tuning large language models typically demands expensive GPUs and long training times. This Skill provides comprehensive guidance for using Unsloth, an open-source framework that trains models 2-5x faster with up to 70-80% less VRAM, making local fine-tuning accessible on consumer hardware. ## Core Features & Use Cases - LoRA/QLoRA Fine-Tuning: Train Llama, Mistral, Gemma, Qwen, Phi, gpt-oss, and DeepSeek models with 4-bit or 16-bit adapters using FastLanguageModel and TRL. - Reinforcement Learning: Run GRPO, GSPO, DPO, ORPO, and KTO training for reasoning models with memory-efficient implementations. - Multimodal Support: Fine-tune vision models, text-to-speech (TTS), speech-to-text (Whisper), and BERT architectures. - Model Export: Save fine-tuned models to GGUF for Ollama and llama.cpp, or 16-bit for vLLM and SGLang deployment. - Use Case: Fine-tune Llama 3.1 (8B) on a custom dataset using a free Colab notebook with QLoRA, then export to GGUF and run it locally in Ollama as a personalized assistant. ## Quick Start Ask the agent to help you fine-tune a Llama model on your dataset using Unsloth with QLoRA and export the result to GGUF for Ollama.

Frequently Asked Questions about unsloth

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with Unsloth?▼

Load the model with FastLanguageModel.from_pretrained using load_in_4bit for QLoRA, attach LoRA adapters, prepare your dataset with the datasets library, and train via the TRL SFTTrainer. Free Colab notebooks cover Llama 3.1, 3.2, and Llama 4 step by step.

What is the difference between LoRA and QLoRA fine-tuning?▼

LoRA trains small 16-bit adapter matrices while freezing the base model, reducing trainable parameters. QLoRA additionally quantizes base weights to 4-bit, cutting VRAM dramatically. Unsloth recommends starting with QLoRA since its dynamic 4-bit quants have negligible accuracy loss.

How much VRAM do I need for Unsloth fine-tuning?▼

QLoRA 4-bit training needs about 5 GB for a 7B model and 41 GB for a 70B model, while 16-bit LoRA needs roughly 19 GB and 164 GB respectively. If you hit out-of-memory errors, lower the batch size to 1, 2, or 3.

Does Unsloth work on Windows or AMD GPUs?▼

Unsloth supports Linux natively and Windows via WSL or a dedicated installation guide. AMD and Intel GPUs are supported, NVIDIA GPUs require CUDA capability 7.0 or higher, and Apple Silicon support is in progress.

How do I run a fine-tuned Unsloth model in Ollama?▼

Save the fine-tuned model to GGUF format using Unsloth's built-in export, which merges adapters into 16-bit weights before quantization. The resulting GGUF file runs directly in Ollama, llama.cpp, Open WebUI, or LM Studio.

Can Unsloth train reasoning models with reinforcement learning?▼

Yes, Unsloth supports GRPO, GSPO, DPO, ORPO, and KTO reinforcement learning with up to 80% less VRAM than other libraries. Notebooks demonstrate training reasoning models like DeepSeek-R1 style Qwen3 and gpt-oss on tasks such as the 2048 game.

Related Skills

Explore More Software Engineering Skills→
cuba6112cuba6112

unsloth-datasets

Standardize ShareGPT and Alpaca datasets for Unsloth fine-tuning.

Community
Intermediate
cryptopaficryptopafi

unsloth

Guide LoRA and QLoRA fine-tuning of LLMs with Unsloth.

Community
Advanced
JeffallanJeffallan
10.9k

fine-tuning-expert

Automate fine-tuning workflows for large language models using LoRA/QLoRA and PEFT methods.

Community
Advanced
BagelHoleBagelHole
46

llm-fine-tuning

Set up QLoRA, LoRA, and full fine-tuning infrastructure with Hugging Face TRL and Axolotl.

Community
Advanced