skills.rest
loteiron
loteironCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

axolotl

Configure and run LLM fine-tuning jobs with Axolotl YAML configs.

Updated Sep 10, 2026
Search Tags:#HuggingFace#LoRA#LLM fine-tuning#GRPO#dpo#axolotl#YAML config
One-click install
npx skills add https://github.com/loteiron/ZeusAgent --skill axolotl-loteiron
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: axolotl
Source: https://github.com/loteiron/ZeusAgent/tree/main/optional-skills/mlops/training/axolotl
Command: npx skills add https://github.com/loteiron/ZeusAgent --skill axolotl-loteiron

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires axolotl, torch, transformers, datasets, peft, accelerate, deepspeed, and includes references (resource) components.

What problem does it solve? Fine-tuning large language models involves complex configuration across training methods, dataset formats, distributed setups, and quantization, and this Skill provides expert guidance for doing it correctly with Axolotl. ## Core Features & Use Cases - YAML-Based Fine-Tuning: Configure LoRA, QLoRA, DPO, KTO, ORPO, and GRPO training runs through declarative Axolotl config files. - Dataset Format Guidance: Choose and configure pre-training, conversation (chat_template), instruction, template-free, and preference dataset formats. - Distributed & Optimized Training: Set up FSDP, DeepSpeed, context parallelism, mixed precision (BF16/FP8), and LoRA kernel optimizations. - Use Case: You want to fine-tune a Llama model on a custom chat dataset with QLoRA on multiple GPUs. This Skill helps you write the YAML config, map your dataset roles, and enable FSDP with the correct transformer layer wrapping. ## Quick Start Ask the agent to write an Axolotl YAML config for QLoRA fine-tuning of a Llama model on your chat dataset.

Frequently Asked Questions about axolotl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune an LLM with Axolotl?▼

Fine-tuning with Axolotl starts by writing a YAML config specifying the base model, adapter type such as lora or qlora, datasets, and training hyperparameters. You then run the axolotl train CLI command with that config file.

What dataset formats does Axolotl support?▼

Axolotl supports pre-training text datasets, conversation datasets using chat_template, instruction datasets like Alpaca, template-free input_output formats, and preference formats for DPO, KTO, ORPO, and GRPO. Datasets can load from JSONL, CSV, Parquet, or the Hugging Face hub.

Does Axolotl support multi-GPU training with FSDP?▼

Axolotl supports FSDP, DeepSpeed, and context parallelism for multi-GPU training. You enable FSDP by setting fsdp_version and fsdp_config in the YAML, including the transformer layer class to wrap, such as LlamaDecoderLayer.

How do I merge LoRA adapters after Axolotl training?▼

Use the axolotl merge-lora CLI command with your training config to merge LoRA adapters into the base model. Using the official command avoids vocab size mismatch errors because Axolotl expands model embeddings when the tokenizer has more tokens.

Why does Axolotl training fail with a padding token error?▼

This error occurs when the tokenizer lacks a padding token. Fix it by adding a pad_token under the special_tokens section of your YAML config, typically set to the same value as the eos_token.

Can Axolotl train multimodal vision language models?▼

Axolotl has beta support for multimodal models including Llama 3.2 Vision, Llama 4, Pixtral, and Llava. It requires setting processor_type, disabling sample packing, and using an extended chat_template dataset format with image content keys.

Related Skills

Explore More Software Engineering Skills→
twjarviszyra-webtwjarviszyra-web
3

axolotl

Configure Axolotl YAML for LLM fine-tuning and evaluation workflows.

Community
Advanced
MedTiLabMedTiLab
5

peft-fine-tuning

Fine-tune large language models with LoRA, QLoRA, and adapter methods using HuggingFace PEFT.

Community
Advanced
HexalithHexalith

bmad-eval-runner

Run skill evals across baseline, variant, quality, and trigger modes and report results.

Official
Advanced
HuskyDannyHuskyDanny

unsloth

Fine-tune LLMs with LoRA and QLoRA using the Unsloth framework.

Community
Advanced