skills.rest
xu1713
xu1713COMMUNITY
Explore all skills by this author →
GitHub

unsloth

Fine-tune LLMs with LoRA and QLoRA using Unsloth's optimized training framework.

Updated Jun 5, 2026
Search Tags:#fine-tuning#lora#qlora#gguf#grpo#unsloth#llm training
One-click install
npx skills add https://github.com/xu1713/openhorse --skill unsloth-xu1713
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: unsloth
Source: https://github.com/xu1713/openhorse/tree/main/openhorse/openhorse/optional-skills/mlops/training/unsloth
Command: npx skills add https://github.com/xu1713/openhorse --skill unsloth-xu1713

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires unsloth, torch, transformers, trl, datasets, peft, and includes references (resource) components.

What problem does it solve? Training large language models typically requires expensive GPU clusters and long training times. This Skill provides guidance for using Unsloth, an open-source framework that fine-tunes LLMs 2-5x faster with up to 70-80% less VRAM, making local fine-tuning feasible on consumer GPUs, Colab, and Kaggle. ## Core Features & Use Cases - LoRA/QLoRA Fine-Tuning: Train models like Llama, Qwen, Gemma, Mistral, Phi, and gpt-oss with 4-bit or 16-bit adapters using FastLanguageModel and TRL's SFTTrainer. - Reinforcement Learning: Run GRPO, GSPO, DPO, ORPO, and KTO training, including vision RL and reasoning model creation. - Model Export & Deployment: Save fine-tuned models to GGUF for Ollama and llama.cpp, or 16-bit formats for vLLM deployment. - Use Case: Fine-tune Llama 3.1 (8B) on a custom dataset using a free Colab GPU with QLoRA, then export to GGUF and run it locally in Ollama. ## Quick Start Ask the assistant to show how to fine-tune a Llama model on a custom dataset using Unsloth with 4-bit quantization.

Frequently Asked Questions about unsloth

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with Unsloth?▼

Load the model with FastLanguageModel.from_pretrained using load_in_4bit=True, attach LoRA adapters with get_peft_model, then train with TRL's SFTTrainer on your dataset. Unsloth provides free Colab notebooks for Llama, Qwen, Gemma, and other models.

What is the difference between LoRA and QLoRA fine-tuning?▼

LoRA trains small adapter matrices in 16-bit precision while freezing the base model. QLoRA additionally quantizes the base model to 4-bit, cutting VRAM usage dramatically. Unsloth recommends starting with QLoRA since its dynamic 4-bit quants have negligible accuracy loss.

How much VRAM do I need for Unsloth fine-tuning?▼

QLoRA 4-bit fine-tuning needs about 5 GB for a 7B model and 41 GB for a 70B model, while 16-bit LoRA needs roughly 19 GB for 7B. If you run out of memory, lower the batch size to 1, 2, or 3.

Does Unsloth work on Windows and AMD GPUs?▼

Yes. Unsloth supports Linux and Windows (including WSL), and works on NVIDIA GPUs with CUDA capability 7.0+, plus AMD and Intel GPUs. The simplest setup is pip install unsloth or the official unsloth/unsloth Docker image.

How do I run my fine-tuned model in Ollama or vLLM?▼

Save the fine-tuned model to GGUF format for Ollama, llama.cpp, and Open WebUI, or merge to 16-bit for vLLM deployment. Unsloth's saving utilities handle the conversion directly after training.

Can Unsloth train vision or text-to-speech models?▼

Yes. Unsloth supports vision/multimodal fine-tuning for models like Qwen3-VL and Gemma 3, TTS models like Orpheus and Sesame-CSM, BERT, and reinforcement learning including vision RL with GRPO and GSPO.

Related Skills

Explore More Software Engineering Skills→
cuba6112cuba6112

unsloth-datasets

Standardize ShareGPT and Alpaca datasets for Unsloth fine-tuning.

Community
Intermediate
OpenLAIROpenLAIR
1.0k

unsloth

Guide memory-efficient LLM fine-tuning with Unsloth using 4-bit quantization.

Official
Intermediate
JeffallanJeffallan
10.9k

fine-tuning-expert

Automate fine-tuning workflows for large language models using LoRA/QLoRA and PEFT methods.

Community
Advanced
BagelHoleBagelHole
46

llm-fine-tuning

Set up QLoRA, LoRA, and full fine-tuning infrastructure with Hugging Face TRL and Axolotl.

Community
Advanced