skills.rest
Jensen-Yao
Jensen-YaoCOMMUNITY
Explore all skills by this author →
GitHub

unsloth

Guides LLM fine-tuning with Unsloth using LoRA, QLoRA, and GRPO techniques.

2|Updated Aug 15, 2026
Search Tags:#llm#fine-tuning#GGUF#LoRA#GRPO#QLoRA#Unsloth
One-click install
npx skills add https://github.com/Jensen-Yao/agents-skills --skill unsloth-jensen-yao
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: unsloth
Source: https://github.com/Jensen-Yao/agents-skills/tree/main/skills/unsloth
Command: npx skills add https://github.com/Jensen-Yao/agents-skills --skill unsloth-jensen-yao

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires unsloth, torch, transformers, trl, datasets, peft, and includes references (resource) components.

What problem does it solve? Fine-tuning large language models typically demands excessive VRAM and slow training runs, blocking developers with consumer GPUs from customizing models like Llama, Qwen, Gemma, or Mistral for their own tasks. ## Core Features & Use Cases - Memory-Efficient Fine-Tuning: Apply LoRA and QLoRA (4-bit) training to fit models from 3B to 70B+ parameters on limited GPU memory, with VRAM requirement tables per model size. - Reinforcement Learning Training: Train reasoning models with GRPO, GSPO, DPO, ORPO, and KTO using up to 80% less VRAM. - Model Export & Deployment: Save fine-tuned models to GGUF, Ollama, vLLM, or SGLang formats for local inference and serving. - Use Case: Fine-tune Llama 3.1 (8B) on a custom dataset in a free Colab notebook, then export it to GGUF and run it locally through Ollama. ## Quick Start Ask the agent to help you fine-tune a Llama or Qwen model with Unsloth using QLoRA on your available GPU.

Frequently Asked Questions about unsloth

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with Unsloth?▼

Load the model with FastLanguageModel.from_pretrained, attach LoRA adapters, prepare your dataset with the datasets library, and train using the TRL SFTTrainer. Unsloth provides free Colab notebooks for Llama, Qwen, Gemma, and other models.

How much VRAM do I need for QLoRA fine-tuning?▼

QLoRA (4-bit) requires about 3.5 GB for 3B models, 5 GB for 7B, 6 GB for 8B, and 41 GB for 70B models. LoRA (16-bit) needs significantly more, such as 19 GB for 7B and 164 GB for 70B.

LoRA vs QLoRA: which one should I use?▼

QLoRA is recommended as the starting point because it combines LoRA adapters with 4-bit quantization, dramatically lowering memory usage. With Unsloth's dynamic 4-bit quants, accuracy loss versus 16-bit LoRA is negligible.

Does Unsloth work on Windows or AMD GPUs?▼

Unsloth supports Linux and Windows (including WSL), plus NVIDIA GPUs with CUDA capability 7.0 or higher. AMD and Intel GPUs are also supported, while Apple Silicon support is in progress.

Why does my Unsloth training run out of memory?▼

Out-of-memory errors usually come from setting the batch size too high. Lower it to 1, 2, or 3 to reduce VRAM consumption, and check the VRAM requirement table for your model size and fine-tuning method.

Can I run my fine-tuned model in Ollama or vLLM?▼

Yes, Unsloth can save fine-tuned models to GGUF format for Ollama, llama.cpp, and Open WebUI, or merge them to 16-bit for vLLM and SGLang deployment. Guides cover saving, exporting, and troubleshooting inference.

Related Skills

Explore More Software Engineering Skills→
cuba6112cuba6112

unsloth-datasets

Standardize ShareGPT and Alpaca datasets for Unsloth fine-tuning.

Community
Intermediate
cuba6112cuba6112

unsloth-gguf

Export fine-tuned machine learning models to GGUF format with quantization and LoRA merging.

Community
Intermediate
JeffallanJeffallan
10.9k

fine-tuning-expert

Automate fine-tuning workflows for large language models using LoRA/QLoRA and PEFT methods.

Community
Advanced
JKhyroJKhyro

peft-fine-tuning

Fine-tune large language models with parameter-efficient adapters like LoRA and QLoRA.

Community
Advanced