simpo-training

SimPO aligns LLMs using preference data without needing a reference model. It supports configurable beta and gamma hyperparameters, optional SFT regularization, and common loss types to optimize instruction following and reasoning across various model sizes.

SimPO enables LLM alignment without a reference model by applying preference optimization directly to human preference data. You configure hyperparameters like beta and gamma, select loss types, and optionally apply SFT regularization to fine-tune base models for instruction following.

Yes, SimPO works with Ultrafeedback-style preference datasets for fine-tuning base models. It supports common loss types and dataset configurations, allowing you to improve LLM alignment and instruction following using standard human preference data formats.

Yes, you can configure beta and gamma hyperparameters for preference learning in SimPO. The method supports these adjustable parameters along with optional SFT regularization and common loss types to control the reference-free alignment optimization process across different model sizes.

SimPO is a reference-free alternative to DPO that eliminates the need for a reference model during preference optimization. Unlike standard DPO approaches, SimPO relies on configurable beta and gamma parameters to achieve strong alignment performance using only human preference data.