skills.rest
kaminocorp
kaminocorpOFFICIAL
Explore all skills by this author →
Website𝕏@kaminocorpoGitHub

simpo-training

Configure reference-free preference optimization for LLM alignment with adjustable beta, gamma, and SFT weight.

228|35|Updated Mar 9, 2026
Search Tags:#llm#alignment#training#simpo#preference#reference-free
One-click install
npx skills add https://github.com/kaminocorp/hermes-alpha --skill simpo-training-kaminocorp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: simpo-training
Source: https://github.com/kaminocorp/hermes-alpha/tree/main/hermes-agent/skills/mlops/training/simpo
Command: npx skills add https://github.com/kaminocorp/hermes-alpha --skill simpo-training-kaminocorp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, transformers, datasets, trl, accelerate, and includes references (resource) components.

What problem does it solve?

SimPO provides a reference-free preference optimization method for aligning LLMs, offering an efficient alternative to DPO with no need for a reference model.

Core Features & Use Cases

  • Reference-free optimization for preference learning across model sizes (7B, 8B, and instruct variants) and diverse datasets.
  • Configurable hyperparameters (beta, gamma, learning rate) and SFT regularization to balance alignment with capability preservation.
  • Use cases include training and fine-tuning for instruction-following, reasoning, and general alignment tasks across varying domains.

Quick Start

Install the required dependencies and launch the training with a suitable configuration file using Accelerate.

Frequently Asked Questions about simpo-training

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I align LLMs with preference optimization without a reference model?▼

Reference-free preference optimization aligns LLMs without a reference model by applying SimPO with configurable beta, gamma, and SFT weight to balance alignment with capability preservation across diverse datasets.

What is the difference between SimPO and DPO for LLM alignment?▼

SimPO offers a reference-free alternative to DPO, removing the need for a reference model while still enabling effective preference learning and instruction-following alignment across 7B and 8B model sizes.

How do I use SimPO to train a 7B or 8B LLM with a custom preference dataset?▼

Configure end-to-end training using Accelerate with your custom dataset, adjusting hyperparameters like beta, gamma, and learning rate. The implementation supports dataset handling and optional SFT regularization to fine-tune instruct variants.

Can I use SimPO with PyTorch and Transformers for instruction-following tasks?▼

Yes, SimPO is built on PyTorch, Transformers, and TRL, supporting end-to-end training configurations for instruction-following, reasoning, and general alignment tasks across varying domains and model sizes.

When should I use SFT regularization during preference optimization?▼

SFT regularization should be applied when you need to balance preference alignment with preserving the base model's original capabilities, preventing capability loss during the reference-free optimization process.

What loss functions are supported for reference-free LLM training?▼

SimPO supports both sigmoid and hinge loss function choices for reference-free LLM preference optimization, allowing you to tailor the alignment training to specific dataset characteristics and model requirements.

Related Skills

Explore More Software Engineering Skills→
MesferAliMesferAli

simpo-training

Implement SimPO training for LLM alignment using PyTorch and Transformers.

Community
Advanced
projectedanxprojectedanx
1

fine-tuning-with-trl

Implement SFT, DPO, PPO, and GRPO post-training alignment with HuggingFace TRL.

Community
Advanced
ricablericable

fine-tuning-with-trl

Fine-tune large language models with TRL for supervised and preference alignment.

Community
Advanced
markwang2658markwang2658
31

simpo-training

Optimize LLM preferences without a reference model using SimPO training.

Community
Intermediate