huggingface-llm-trainer

Automate TRL training workflows on Hugging Face Jobs by using inline templates that manage setup, monitoring, and deployment. The process covers SFT, DPO, and GRPO methods while integrating Trackio for monitoring and automating hub pushes.

Unsloth is supported for memory-efficient fine-tuning alongside standard TRL methods. This enables training large language models with reduced VRAM overhead while maintaining hub authentication and Trackio monitoring integration.

Export fine-tuned models to GGUF format for local deployment using the built-in conversion workflow. This step follows the TRL or Unsloth training phase and prepares the model for offline inference environments.

The workflow supports training for both language and vision models. It applies TRL methods including SFT, DPO, and GRPO, as well as reward modeling, across various model architectures.

Monitor Hugging Face training jobs and estimate costs using integrated Trackio monitoring and provided cost estimation guidelines. The templates include hardware selection criteria to balance performance requirements against cloud training expenses.

Dependencies including trl, peft, transformers, accelerate, and unsloth are required to run the training jobs. The environment also uses trackio for monitoring, datasets for data handling, and huggingface_hub for authentication and model persistence.