skills.rest
MedTiLab
MedTiLabCOMMUNITY
Explore all skills by this author →
GitHub

axolotl

Configure and fine-tune LLMs with Axolotl YAML configs, LoRA, DPO, and multimodal training.

5|2|Updated Aug 10, 2026
Search Tags:#llm#fine-tuning#HuggingFace#LoRA#dpo#axolotl#YAML config
One-click install
npx skills add https://github.com/MedTiLab/Auto-meta-analysis --skill axolotl-medtilab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: axolotl
Source: https://github.com/MedTiLab/Auto-meta-analysis/tree/main/skills/fine-tuning/axolotl
Command: npx skills add https://github.com/MedTiLab/Auto-meta-analysis --skill axolotl-medtilab

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Fine-tuning large language models requires navigating complex training configurations, dataset formats, and distributed training setups. This Skill provides expert guidance on Axolotl, the YAML-driven fine-tuning framework, so you can configure SFT, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, and multimodal training without digging through scattered documentation. ## Core Features & Use Cases - YAML Configuration Guidance: Get correct config patterns for FSDP, DeepSpeed, mixed precision (FP16/BF16/FP8), sample packing, and context parallelism across 100+ model architectures. - Dataset Format Selection: Choose and configure the right dataset format (pre-training, chat_template, instruction, template-free input_output, pre-tokenized) with masking and role-mapping rules. - RLHF & Preference Training: Set up DPO, IPO, ORPO, KTO, GRPO, reward modeling, and process reward models with the correct dataset schemas and TRL integrations. - Use Case: You want to QLoRA fine-tune Llama 3 on a custom chat dataset. Ask for the config, and get a complete YAML with chat_template settings, roles_to_train masking, LoRA target modules, and 4-bit quantization. ## Quick Start Ask the AI to generate an Axolotl YAML config for fine-tuning a specific model on your dataset format, such as a ChatML conversation dataset with LoRA.

Frequently Asked Questions about axolotl

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fine-tune a Llama model with Axolotl?▼

Create a YAML config specifying base_model, adapter type (lora or qlora), datasets with their format type, and training hyperparameters, then run axolotl train config.yml. Axolotl supports 100+ model architectures including Llama, Mistral, Gemma, and Qwen.

What dataset formats does Axolotl support for fine-tuning?▼

Axolotl supports pre-training text, chat_template conversation formats (OpenAI and ShareGPT styles), instruction formats like Alpaca, template-free input_output with segment masking, and pre-tokenized datasets. Preference methods like DPO use chosen/rejected pair formats.

Does Axolotl support DPO and GRPO training?▼

Yes, Axolotl wraps the TRL library to support DPO, IPO, ORPO, KTO, GRPO, and reward modeling. GRPO uses vLLM for accelerated trajectory generation and requires installing axolotl with the vllm extra.

How do I merge LoRA adapters in Axolotl?▼

Use the axolotl merge-lora CLI command with your training config rather than custom scripts. This avoids vocab size mismatch errors, since Axolotl expands model embeddings when the tokenizer has more tokens than the base model.

Why does Axolotl training hang or stop progressing on multiple GPUs?▼

This is usually a GPU communication issue related to NCCL. Run NCCL tests like all_reduce_perf to pinpoint bandwidth bottlenecks, and verify your FSDP or DeepSpeed configuration matches your GPU topology.

Can Axolotl train multimodal vision-language models?▼

Yes, Axolotl has beta support for multimodal models including Llama 3.2 Vision, Llama 4, Pixtral, and Llava-1.5 using an extended chat_template format. Sample packing is not yet supported, and feature parity with text training is limited.

Related Skills

Explore More Software Engineering Skills→
CHENyiru3CHENyiru3

datasets

Load and preprocess datasets from Hugging Face Hub or local sources.

Community
Advanced
cuba6112cuba6112

unsloth-gguf

Export fine-tuned machine learning models to GGUF format with quantization and LoRA merging.

Community
Intermediate
HexalithHexalith

bmad-eval-runner

Run skill evals across baseline, variant, quality, and trigger modes and report results.

Official
Advanced
matlee0409matlee0409

unsloth

Automate LoRA/QLoRA fine-tuning of LLMs with Unsloth on local GPUs.

Community
Advanced