simpo-training

Name: simpo-training
Availability: InStock
Author: monjyu1101

Run SimPO training scripts to align LLMs without a reference model.

1|Updated Jan 31, 2026

Search Tags:#mlops #simpo #reference-free #preference-optimization #training-config #llm-alignment #dataset-handling

Ready to run

npx skills add https://github.com/Monjyu1101/AiDiy2026 --skill simpo-training-monjyu1101

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: simpo-training
Source: https://github.com/Monjyu1101/AiDiy2026/tree/main/backend_hermes/optional-skills/mlops/simpo
Command: npx skills add https://github.com/Monjyu1101/AiDiy2026 --skill simpo-training-monjyu1101

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

SimPO provides a reference-free approach to align LLMs by learning preferences without a ground-truth or reference model, enabling more scalable and efficient alignment.

Core Features & Use Cases

Reference-free optimization for language-model alignment
Supports training from base models, mixed datasets, and various model sizes
Applicable to instruction-following, reasoning, and chat tasks in single-node or distributed configurations

Quick Start

Install dependencies, prepare your data, and run the recommended training command to start SimPO.

Frequently Asked Questions about simpo-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I align an LLM without using a reference model?▼

Reference-free preference optimization aligns LLMs without a reference model by learning preferences directly. SimPO achieves this by using a specific configuration with beta, gamma_beta_ratio, sft_weight, and loss_type to optimize models from 7B to 70B.

What is SimPO and how does it work for preference optimization?▼

SimPO is a reference-free preference optimization method that aligns LLMs without a ground-truth or reference model. It enables more scalable and efficient alignment by learning preferences directly from a base model and a preference dataset.

Can I train a 70B model with SimPO on a distributed setup?▼

Yes, SimPO supports training models from 7B to 70B across single-node or distributed configurations. It is applicable to instruction-following, reasoning, and chat tasks using standard datasets.

How do I start SimPO training for instruction-following tasks?▼

To start SimPO training, install Python and PyTorch dependencies, prepare a base model and a preference dataset, configure the SimPO parameters, and run the recommended training script to begin reference-free optimization.

Does reference-free LLM alignment require SFT regularization?▼

SFT regularization is optional in SimPO. You can configure it alongside the core SimPO parameters including beta, gamma_beta_ratio, and loss_type to balance preference learning with supervised fine-tuning.

Why use reference-free preference optimization over traditional methods?▼

Reference-free preference optimization eliminates the need for a reference model during LLM alignment, enabling more scalable and efficient training. This approach simplifies the pipeline while maintaining alignment for instruction-following and reasoning tasks.