skills.rest
Chris-Chai-Minjae
Chris-Chai-MinjaeCOMMUNITY
Explore all skills by this author →
GitHub

simpo-training

Train LLMs with reference-free SimPO preference optimization via YAML configs.

Updated Apr 23, 2026
Search Tags:#fine-tuning#pytorch#transformers#simpo#reference-free#preference-optimization#llm-alignment
One-click install
npx skills add https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge --skill simpo-training-chris-chai-minjae
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: simpo-training
Source: https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge/tree/main/optional-skills/mlops/simpo
Command: npx skills add https://github.com/Chris-Chai-Minjae/hermes-agent-r1-bridge --skill simpo-training-chris-chai-minjae

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

SimPO provides a reference-free approach to optimize language-model preferences, enabling faster, safer alignment without needing a base reference model.

Core Features & Use Cases

  • Reference-free alignment improvement: replaces the need for a reference model while preserving strong performance.
  • Flexible experimentation: supports various model sizes and datasets with config-driven workflows.
  • Practical use: training/instruction-following alignment with SimPO on standard benchmarks.

Quick Start

Configure a SimPO experiment with a YAML file and run the training script to begin alignment.

Frequently Asked Questions about simpo-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How does reference-free preference optimization work for LLM alignment?▼

Reference-free preference optimization aligns LLMs without a base reference model by using SimPO configurations to directly optimize preferences, improving training efficiency and reducing memory overhead.

How do I configure SimPO for fine-tuning language models?▼

Configure SimPO fine-tuning by creating a YAML file specifying the model, dataset_mixer, learning_rate, beta, gamma_beta_ratio, loss_type, and optional sft_weight, then run the training script.

Can I use SimPO to align 7B to 70B models with standard preference datasets?▼

Yes, SimPO supports instruction-following and reasoning alignment for model sizes from 7B to 70B using standard preference datasets and requires PyTorch, transformers, datasets, TRL, and accelerate.

What is the best way to optimize LLM preferences without a reference model in PyTorch?▼

SimPO provides a reference-free approach to optimize LLM preferences in PyTorch, replacing the need for a base reference model while preserving strong performance on instruction-following tasks.

Why use SimPO instead of standard RLHF for preference optimization?▼

SimPO eliminates the reference model requirement in standard RLHF, enabling faster and safer alignment while maintaining strong performance across various model sizes and standard preference datasets.

Related Skills

Explore More Data & Analytics Skills→
samuelmukotisamuelmukoti

simpo-training

Optimize LLM alignment preferences without a reference model.

Community
Intermediate
Yangel-hideYangel-hide

simpo-training

Optimize LLM preferences with SimPO using chosen versus rejected responses.

Community
Advanced
KarlinskySKarlinskyS

simpo-training

Train 7B-70B LLMs with reference-free SimPO preference optimization.

Community
Intermediate
Orchestra-ResearchOrchestra-Research
11.5k

simpo-training

Train large language models with SimPO preference optimization without a reference model.

Official
Intermediate