hugging-face-model-trainer

You can fine-tune an LLM on Hugging Face Jobs by using TRL library workflows on managed cloud infrastructure. This approach eliminates local GPU setup by handling environment management and executing training directly on Hugging Face Jobs.

Yes, Hugging Face Jobs supports DPO and GRPO training methods through the TRL library integration. It also facilitates supervised fine-tuning (SFT) and reward modeling for language models on the cloud infrastructure.

Yes, you can convert a fine-tuned model to GGUF for local deployment with Ollama. The workflow includes integrated guidance for converting trained models into the GGUF format after the training process completes.

The best way to monitor LLM training costs and metrics on Hugging Face Jobs is by using Trackio. The workflow provides integrated guidance on hardware selection, cost estimation, and metrics monitoring via Trackio throughout the training lifecycle.

Yes, you need to prepare your dataset differently for SFT versus DPO. The workflow provides integrated guidance on dataset preparation tailored to the specific training method, ensuring compatibility with the TRL library.