simpo-training

Optimize LLM alignment preferences without a reference model.

Updated Aug 27, 2026

Search Tags:#training #datasets #simpo #ml-ops #reference-free #preference-optimization #llm-alignment

Ready to run

npx skills add https://github.com/wwwillott/jobnimbus --skill simpo-training-wwwillott

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: simpo-training
Source: https://github.com/wwwillott/jobnimbus/tree/main/optional-skills/mlops/simpo
Command: npx skills add https://github.com/wwwillott/jobnimbus --skill simpo-training-wwwillott

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

SimPO provides a reference-free method to align LLMs by learning preferences directly from data, removing the need for a separate reference model.

Core Features & Use Cases

Reference-free optimization enabling efficient, direct preference learning for model alignment.
Flexible workflows for base-model training, instruct-finetuning, and reasoning tasks across model families.
Configurable hyperparameters (beta, gamma, gamma_beta_ratio, sft_weight, loss_type) and dataset integration for various domains.

Quick Start

Run the training script with your chosen base model and dataset to start SimPO training.

Frequently Asked Questions about simpo-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

What is reference-free LLM preference optimization?▼

Reference-free LLM preference optimization aligns models by learning preferences directly from data, removing the need for a separate reference model. SimPO satisfies this loss objective to enable efficient, direct preference learning for model alignment.

How do I apply SimPO for instruct fine-tuning on Llama-3-8B-Instruct?▼

To apply SimPO for instruct fine-tuning, run the training script with your base model and dataset. It supports configurable hyperparameters like beta, sft_weight, and loss_type for dataset-driven workflows across architectures like Llama-3-8B-Instruct.

Can I use reference-free preference optimization for reasoning tasks?▼

Yes, reference-free preference optimization supports reasoning tasks. The method applies to base-model training, instruct fine-tuning, and reasoning workflows across various model families such as Mistral-7B.

Do I need a separate reference model for LLM alignment?▼

No, you do not need a separate reference model for LLM alignment. SimPO provides a reference-free method that learns preferences directly from your datasets, streamlining the optimization process.

What hyperparameters are configurable for reference-free preference learning?▼

Configurable hyperparameters for reference-free preference learning include beta, gamma_beta_ratio, sft_weight, and loss_type. These settings allow flexible, dataset-driven workflows across various domains.