skills.rest
cuba6112
cuba6112COMMUNITY
Explore all skills by this author →
GitHub

unsloth-gguf

Export fine-tuned machine learning models to GGUF format with quantization and LoRA merging.

Updated Aug 23, 2026
Search Tags:#quantization#ollama#gguf#llama.cpp#unsloth#model export
One-click install
npx skills add https://github.com/cuba6112/skillfactory --skill unsloth-gguf
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: unsloth-gguf
Source: https://github.com/cuba6112/skillfactory/tree/main/skills/unsloth-gguf
Command: npx skills add https://github.com/cuba6112/skillfactory --skill unsloth-gguf

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill simplifies the process of exporting fine-tuned machine learning models into the GGUF format, making them compatible with popular local inference tools like llama.cpp and Ollama.

Core Features & Use Cases

  • GGUF Export: Converts trained models to the efficient GGUF format.
  • Quantization: Supports various quantization methods (e.g., q4_k_m, q8_0) to reduce model size and VRAM usage.
  • LoRA Merging: Automatically merges LoRA adapters into the base model during export.
  • Use Case: Deploy a fine-tuned LLM on your local machine for faster inference or to run it on hardware with limited VRAM.

Quick Start

Export the current model to GGUF format using the 'q4_k_m' quantization method.

Frequently Asked Questions about unsloth-gguf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I export a fine-tuned model to GGUF format for local inference?▼

You can export your fine-tuned model to GGUF format by specifying a quantization method like q4_k_m, which converts the model and merges any LoRA adapters into the base model for optimized local inference.

What quantization methods are supported when exporting models to GGUF?▼

The GGUF export process supports various quantization methods, including q4_k_m and q8_0, which reduce model size and VRAM usage to enable deployment on hardware with limited memory resources.

Can I merge LoRA adapters into a base model during GGUF export?▼

Yes, you can merge LoRA adapters into the base model during GGUF export. This Skill automatically integrates your trained adapters into the base model during the conversion process.

Does the exported GGUF model work with Ollama and llama.cpp?▼

Yes, the exported GGUF model is explicitly designed to be compatible with local inference engines. It enables direct deployment and serving using tools like llama.cpp and Ollama.

Why do I need to quantize my model to GGUF format for local deployment?▼

Quantizing your model to GGUF format reduces its file size and VRAM requirements, making it feasible to run large language models locally on consumer hardware with limited memory capacity.

What is the best way to deploy a fine-tuned LLM on a machine with limited VRAM?▼

The best way to deploy on limited VRAM is exporting the fine-tuned model to GGUF format using q4_k_m quantization, which compresses the model size while retaining compatibility with local serving tools like Ollama.

Related Skills

Explore More Software Engineering Skills→
camelranchentertainmentcamelranchentertainment
1

fine-tuning-expert

Fine-tune language models with LoRA/QLoRA adapters on JSONL datasets.

Community
Advanced
tdiminotdimino
41

llama-cpp

Run GGUF model inference with llama.cpp and apply LoRA adapters.

Community
Advanced
afel6afel6

gguf-quantization

Convert and quantize transformer models into GGUF format for CPU inference.

Community
Advanced
ScientiaCapitalScientiaCapital
1

model-deployment

Export fine-tuned models to GGUF and deploy with vLLM or Ollama.

Community
Advanced