hf-transformers-trainer

What problem does it solve?

Fine-tuning and aligning large language models on local GPU hardware is complex, memory-constrained, and error-prone; this Skill consolidates proven Trainer, PEFT (LoRA/QLoRA), and TRL workflows so you can configure training, avoid OOMs, and run alignment reliably on local machines.

Core Features & Use Cases

• Training configuration templates for TrainingArguments, data collators, schedulers, logging, callbacks, and evaluation to get reproducible Trainer runs.
• PEFT LoRA / QLoRA recipes including BitsAndBytes config, prepare_model_for_kbit_training, recommended r/alpha, and adapter merging patterns.
• TRL alignment workflows for SFT, DPO, and GRPO with dataset format requirements, reward function examples, and validation checks.
• VRAM estimation & optimization: gradient checkpointing, bf16/fp16 guidance, DeepSpeed/FSDP suggestions, packing, and monitoring tips to prevent OOMs.
• Dataset & tokenization patterns for instruction tuning, ChatML/Llama formats, label masking, and preference pair generation.
• Real-world example: run QLoRA + LoRA adapters to SFT a 7B model on an 8–12 GB GPU, then continue with DPO using prepared preference pairs.

Quick Start

Use the hf-transformers-trainer guidance to fine-tune a Llama-family model locally with QLoRA and LoRA adapters while applying gradient checkpointing and recommended TrainingArguments.