simpo-training

SimPO enables reference-free LLM alignment by learning preferences directly without a separate reference model. It works across base and instructional models for alignment fine-tuning, using configurable hyperparameters and optional SFT regularization to balance capability preservation and alignment.

SimPO is a reference-free preference optimization method requiring fewer forward passes than DPO or PPO. This makes it more efficient and scalable for single-node or modest GPU setups compared to DPO or PPO.

SimPO uses YAML configs to manage hyperparameters including beta, gamma, learning_rate, sft_weight, and loss_type to control alignment dynamics. Optional SFT regularization balances capability preservation and alignment during fine-tuning.

Yes, SimPO is efficient and scalable, requiring fewer forward passes than DPO or PPO, making it suitable for single-node or modest GPU setups. You can run the training workflow using included YAML configs and dataset references.

SFT regularization is optional in SimPO, used to balance capability preservation and alignment during preference optimization. It is controlled via the sft_weight hyperparameter within your YAML training configs.

Reference-free preference optimization reduces training complexity by eliminating the need to load a separate reference model. This decreases memory requirements and forward passes, making alignment fine-tuning more efficient.