skills.rest
Chensihakniroth
ChensihaknirothCOMMUNITY
Explore all skills by this author →
GitHub

axolotl

Configure and run YAML-based LLM fine-tuning with LoRA, DPO, and GRPO.

Updated Jun 7, 2026
Search Tags:#LoRA#distributed training#LLM fine-tuning#GRPO#dpo#axolotl#YAML config
One-click install
npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill axolotl-chensihakniroth
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: axolotl
Source: https://github.com/Chensihakniroth/ANAKOT-AGENT/tree/main/optional-skills/mlops/training/axolotl
Command: npx skills add https://github.com/Chensihakniroth/ANAKOT-AGENT --skill axolotl-chensihakniroth

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires axolotl, torch, transformers, datasets, peft, accelerate, deepspeed, and includes references (resource) components.

What problem does it solve? Fine-tuning large language models requires coordinating complex training configurations, dataset formats, and distributed GPU setups, which is error-prone when done manually. This Skill provides expert guidance for the Axolotl framework so you can define training runs declaratively in YAML instead of writing custom training loops. ## Core Features & Use Cases - YAML-Driven Fine-Tuning: Configure LoRA, QLoRA, full fine-tuning, and preference methods (DPO, KTO, ORPO, GRPO) through declarative config files supporting 100+ model architectures. - Dataset Format Guidance: Handle pre-training, instruction, conversation (chat_template), template-free, and pre-tokenized dataset formats with correct masking and tokenization. - Distributed Training & Optimization: Set up FSDP, DeepSpeed, sequence parallelism, mixed precision (BF16/FP8), Flash Attention, and LoRA kernel optimizations for multi-GPU runs. - Use Case: You want to fine-tune a Llama model on your custom chat dataset using QLoRA across 4 GPUs. This Skill helps you write the correct Axolotl YAML config, choose the chat template, map dataset roles, and launch training. ## Quick Start Ask the agent to help you write an Axolotl YAML config to fine-tune your chosen base model on your dataset with LoRA.

Frequently Asked Questions about axolotl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune an LLM with Axolotl?▼

Fine-tuning with Axolotl requires writing a YAML config specifying your base model, dataset paths, adapter type (such as lora or qlora), and training hyperparameters, then running the axolotl train CLI command. The framework handles tokenization, masking, and training loop execution.

What dataset formats does Axolotl support for fine-tuning?▼

Axolotl supports pre-training text, instruction formats like Alpaca, conversation formats using chat templates (ChatML, Gemma, Llama), template-free input_output with segment masking, and pre-tokenized datasets. Choose based on whether your data is raw text, instruction-response pairs, or message lists.

Does Axolotl support multi-GPU and distributed training?▼

Axolotl supports multi-GPU training via FSDP, DeepSpeed, and context parallelism configured directly in the YAML file. You can enable fsdp_version 2 with transformer layer wrapping, or set context_parallel_size as a divisor of your total GPU count.

Can I use Axolotl for DPO or GRPO preference training?▼

Axolotl supports preference-based post-training including DPO, IPO, KTO, ORPO, and GRPO by wrapping the TRL library. GRPO integrates with vLLM for faster trajectory generation, requiring compatible vLLM and torch versions installed as extras.

Why does Axolotl training fail with chat template errors?▼

Chat template errors usually occur when the tokenizer lacks a default template, role mappings are missing, or EOS/EOT tokens are misconfigured. Set chat_template explicitly, define message_property_mappings for your dataset keys, and configure special_tokens to match your template.

What are the limitations of Axolotl multimodal training?▼

Multimodal support is in beta with limited feature parity: sample packing is unsupported, sequence length truncation is not applied, and some architectures like LLaVA and Pixtral cannot mix text-only and image datasets. Supported models include Llama 3.2 Vision, Llama 4, and Mistral variants.

Related Skills

Explore More Software Engineering Skills→
CHENyiru3CHENyiru3

datasets

Load and preprocess datasets from Hugging Face Hub or local sources.

Community
Advanced
igniteenowigniteenow
16

peft-fine-tuning

Fine-tune large language models with LoRA and QLoRA adapters on limited GPU memory.

Community
Advanced
NVIDIANVIDIA
49

sop-cr-finetuning

Launch and monitor Cosmos-Reason2 fine-tuning on augmented video QA datasets.

Official
Advanced
granietgraniet
247

unsloth

Fine-tune large language models with Unsloth using LoRA and QLoRA optimization.

Community
Intermediate