skills.rest
loteiron
loteironCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

unsloth

Fine-tune LLMs with LoRA and QLoRA using Unsloth's optimized training framework.

Updated Sep 10, 2026
Search Tags:#fine-tuning#LLM training#GGUF#LoRA#GRPO#QLoRA#Unsloth
One-click install
npx skills add https://github.com/loteiron/ZeusAgent --skill unsloth-loteiron
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: unsloth
Source: https://github.com/loteiron/ZeusAgent/tree/main/optional-skills/mlops/training/unsloth
Command: npx skills add https://github.com/loteiron/ZeusAgent --skill unsloth-loteiron

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires unsloth, torch, transformers, trl, datasets, peft, and includes references (resource) components.

What problem does it solve? Training large language models typically requires expensive GPU clusters and complex configuration. This Skill provides guidance for fine-tuning LLMs 2-5x faster with up to 70% less VRAM using Unsloth, making local fine-tuning feasible on consumer hardware. ## Core Features & Use Cases - LoRA/QLoRA Fine-Tuning: Train models like Llama, Mistral, Gemma, Qwen, and gpt-oss with 4-bit or 16-bit adapters using FastLanguageModel and TRL. - Reinforcement Learning: Run GRPO, GSPO, DPO, ORPO, and KTO training with up to 80% less VRAM for reasoning model development. - Model Export & Deployment: Save fine-tuned models to GGUF for Ollama and llama.cpp, or 16-bit formats for vLLM and SGLang serving. - Use Case: Fine-tune Llama 3.1 (8B) on a custom dataset using a free Colab notebook, then export to GGUF and run it locally in Ollama as a personalized assistant. ## Quick Start Ask the agent to help you fine-tune a Llama model on your dataset using Unsloth with QLoRA and export the result to GGUF for Ollama.

Frequently Asked Questions about unsloth

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with Unsloth?▼

Load the model with FastLanguageModel.from_pretrained using load_in_4bit for QLoRA, attach LoRA adapters with get_peft_model, then train with TRL's SFTTrainer on your dataset. Free Colab notebooks cover Llama, Qwen, Gemma, and other models end to end.

How much VRAM do I need for QLoRA fine-tuning?▼

QLoRA (4-bit) needs about 5 GB for a 7B model, 6 GB for 8B, and 41 GB for 70B. LoRA (16-bit) requires roughly 19 GB for 7B and 164 GB for 70B. If you hit out-of-memory errors, reduce batch size to 1, 2, or 3.

LoRA vs QLoRA: which one should I use?▼

QLoRA is recommended for most cases because 4-bit quantization dramatically lowers memory use with negligible accuracy loss thanks to Unsloth's dynamic quants. Use 16-bit LoRA when maximum accuracy matters, such as TTS fine-tuning, and you have sufficient VRAM.

Does Unsloth work on Windows or AMD GPUs?▼

Unsloth supports Linux and Windows (including WSL), plus NVIDIA GPUs with CUDA capability 7.0 or higher. AMD and Intel GPUs are also supported, and an official Docker image unsloth/unsloth avoids manual setup entirely.

How do I run my fine-tuned model in Ollama or vLLM?▼

Merge and save the model to 16-bit, then convert to GGUF for Ollama, llama.cpp, or Open WebUI, or save in 16-bit format directly for vLLM and SGLang deployment. The references include saving guides and troubleshooting steps for inference issues.

Can Unsloth train vision or text-to-speech models?▼

Yes, Unsloth supports vision fine-tuning for models like Qwen3-VL and Gemma 3, TTS models such as Orpheus and Sesame-CSM, and even vision reinforcement learning with GRPO. Any model that works in transformers generally works in Unsloth.

Related Skills

Explore More Software Engineering Skills→
cuba6112cuba6112

unsloth-datasets

Standardize ShareGPT and Alpaca datasets for Unsloth fine-tuning.

Community
Intermediate
superfhpsuperfhp

unsloth

Fine-tune LLMs with LoRA and QLoRA using less VRAM and faster training.

Community
Intermediate
davpatel605-beepdavpatel605-beep

evaluating-llms-harness

Evaluate LLMs against standardized benchmarks like MMLU and GSM8K.

Community
Advanced
NVIDIANVIDIA
49

sop-cr-finetuning

Launch and monitor Cosmos-Reason2 fine-tuning on augmented video QA datasets.

Official
Advanced