simpo-training

Optimize large language model preferences using the reference-free SimPO method.

6|3|Updated Jan 29, 2026

Search Tags:#llm #alignment #training #simpo #preference alignment

Ready to run

npx skills add https://github.com/jonnabio/ace-framework --skill simpo-training-jonnabio

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: simpo-training
Source: https://github.com/jonnabio/ace-framework/tree/main/.ace/packs/ai-research/simpo
Command: npx skills add https://github.com/jonnabio/ace-framework --skill simpo-training-jonnabio

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, transformers, datasets, trl, accelerate, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill addresses the challenge of LLM preference alignment by providing a simple and efficient alternative to DPO, with improved performance and no need for a reference model.

Core Features & Use Cases

Preference Optimization: SimPO is a reference-free method for LLM preference alignment, offering better performance than DPO.
Efficiency: No reference model needed, resulting in faster training.
Use Case: Ideal for scenarios where you want a simpler, faster training process than DPO/PPO, with preference alignment capabilities.

Quick Start

Train the SimPO model for preference alignment on your dataset using the provided training commands.

Frequently Asked Questions about simpo-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How does SimPO preference alignment work for LLMs?▼

SimPO is a reference-free method for LLM preference alignment that optimizes model preferences without requiring a separate reference model. It simplifies the training process while delivering improved performance over traditional methods like DPO.

Do I need a reference model for SimPO training?▼

No, SimPO training is entirely reference-free. You do not need a reference model, which results in a faster and simpler preference alignment process compared to traditional DPO or PPO approaches.

What libraries are required to run SimPO preference optimization?▼

SimPO preference optimization requires PyTorch, Transformers, Datasets, TRL, and Accelerate libraries to function. These dependencies provide the necessary environment for training and optimizing large language models.

How do I train a model with SimPO for preference learning?▼

You can train a model for preference alignment using SimPO by executing the provided training commands on your dataset. The Skill includes scripts designed to facilitate the optimization process within the PyTorch and Transformers environment.

SimPO vs DPO: which is better for LLM preference alignment?▼

SimPO offers a simpler and more efficient alternative to DPO for LLM preference alignment. It eliminates the need for a reference model, resulting in faster training while achieving improved performance compared to DPO.

When should I choose SimPO over PPO for LLM training?▼

Choose SimPO over PPO when you need a simpler and faster training process for preference alignment. It is ideal for scenarios where a reference model is unavailable and you want efficient optimization without PPO's complexity.