skills.rest
agtktID
agtktIDCOMMUNITY
Explore all skills by this author →
GitHub

axolotl

Configure and run YAML-based LLM fine-tuning with LoRA, DPO, and GRPO.

1|Updated Aug 6, 2026
Search Tags:#HuggingFace#LoRA#LLM fine-tuning#GRPO#dpo#axolotl#YAML config
One-click install
npx skills add https://github.com/agtktID/indagis-agent --skill axolotl-agtktid
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: axolotl
Source: https://github.com/agtktID/indagis-agent/tree/main/optional-skills/mlops/training/axolotl
Command: npx skills add https://github.com/agtktID/indagis-agent --skill axolotl-agtktid

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires axolotl, torch, transformers, datasets, peft, accelerate, deepspeed, and includes references (resource) components.

What problem does it solve? Fine-tuning large language models requires juggling complex training configurations, dataset formats, distributed training setups, and RLHF methods. This Skill provides expert guidance for the Axolotl framework so you can write correct YAML configs and debug training issues without digging through hundreds of documentation pages. ## Core Features & Use Cases - YAML Training Configuration: Guidance for LoRA/QLoRA, full fine-tuning, FSDP, DeepSpeed, mixed precision (FP16/BF16/FP8), and multimodal model training across 100+ model architectures. - Dataset Format Support: Covers pre-training, instruction (Alpaca), conversation (chat_template/ChatML), template-free input_output, and preference datasets for DPO/KTO/ORPO/GRPO. - API & Troubleshooting Reference: Includes 150 pages of API documentation plus an FAQ covering NCCL issues, tokenizer padding errors, chat template masking, and vLLM integration. - Use Case: You want to fine-tune Llama 3 with QLoRA on a custom ShareGPT-style dataset. The Skill helps you map dataset keys, choose a chat template, configure 4-bit loading, and set up FSDP for multi-GPU training. ## Quick Start Ask the AI to write an Axolotl YAML config for QLoRA fine-tuning of a Llama model on your instruction dataset.

Frequently Asked Questions about axolotl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune an LLM with Axolotl?▼

Fine-tuning with Axolotl works by writing a YAML config specifying the base model, dataset path and type, adapter method like lora or qlora, and training hyperparameters, then running the axolotl train CLI command. No training code needs to be written directly.

What dataset formats does Axolotl support?▼

Axolotl supports pre-training text datasets, instruction formats like Alpaca, conversation formats using chat_template with ChatML or custom Jinja templates, template-free input_output segments, and preference formats for DPO, KTO, ORPO, and GRPO. Datasets load from JSONL, CSV, parquet, or the Hugging Face hub.

Does Axolotl support multi-GPU and distributed training?▼

Axolotl supports multi-GPU training through FSDP version 2, DeepSpeed, and context parallelism configured directly in the YAML file. The context_parallel_size must divide the total GPU count, and FSDP requires specifying the transformer layer class to wrap, such as LlamaDecoderLayer.

Can Axolotl train multimodal or vision-language models?▼

Axolotl has beta multimodal support for architectures including Llama 3.2 Vision, Llama 4, Pixtral, Llava-1.5, Mistral-Small-3.1, and Voxtral. Multimodal training requires setting processor_type, disabling sample packing, and using an extended chat_template dataset format with image, audio, or video content keys.

Why does Axolotl fail with a tokenizer padding token error?▼

This error occurs because the tokenizer lacks a padding token, which is common for Llama-family models. Fix it by adding a pad_token under the special_tokens section of your YAML config, typically setting it to the same value as the eos_token.

How do I merge LoRA adapters after Axolotl training?▼

Merge LoRA adapters into the base model using the axolotl merge-lora CLI command with the same config used for training. Using the built-in command instead of custom scripts avoids vocab size mismatch errors caused by Axolotl's automatic embedding expansion.

Related Skills

Explore More Software Engineering Skills→
lightning-rod-labslightning-rod-labs
57

public-dataset-exploration

Search Kaggle, Hugging Face, and GitHub for raw public datasets.

Official
Basic
Clay-HHKClay-HHK
1

hqq-quantization

Quantize LLM weights to 4/3/2-bit precision without calibration data using HQQ.

Community
Advanced
hung-phanhung-phan
1

experiment-tracking

Track machine learning experiments and manage model lifecycles with MLflow, Weights & Biases, and Trackio.

Community
Advanced
matlee0409matlee0409

unsloth

Automate LoRA/QLoRA fine-tuning of LLMs with Unsloth on local GPUs.

Community
Advanced