skills.rest
thinking-machines-lab
thinking-machines-labOFFICIAL
Explore all skills by this author →
WebsiteGitHub

rlhf

Orchestrates an RLHF workflow with SFT, RM training, and RL via the Tinker API.

4.0k|507|Updated Jul 14, 2025
Search Tags:#fine-tuning#pipeline#machine-learning#rl#rlhf#tinker#reward-model
One-click install
npx skills add https://github.com/thinking-machines-lab/tinker-cookbook --skill rlhf-thinking-machines-lab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: rlhf
Source: https://github.com/thinking-machines-lab/tinker-cookbook/tree/main/.claude/skills/rlhf
Command: npx skills add https://github.com/thinking-machines-lab/tinker-cookbook --skill rlhf-thinking-machines-lab

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This entry point provides a complete RLHF workflow orchestrated through the Tinker API, enabling end-to-end fine-tuning, reward modeling, and reinforcement learning from human feedback in a single, reproducible pipeline.

Core Features & Use Cases

  • SFT Stage: Fine-tune a base model on instruction data to produce a capable policy.
  • Reward Model Stage: Train a reward model from preference comparisons to score outputs.
  • RL Stage: Optimize the SFT policy using RM scores to maximize alignment with human preferences.
  • Use Case: Collaborate with data scientists to deploy an end-to-end RLHF pipeline for custom domain assistants.

Quick Start

Run the built-in RLHF recipe to execute SFT, RM training, and RL steps in sequence.

Frequently Asked Questions about rlhf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run end-to-end RLHF training pipelines for machine learning models?▼

End-to-end RLHF training orchestrates SFT, reward model training, and RL stages in a single pipeline using Tinker API. You execute the built-in recipe to fine-tune a base model, score outputs via preference comparisons, and optimize alignment sequentially.

Can I configure LoRA rank and learning rates for fine-tuning with reinforcement learning?▼

Configuring LoRA rank, base models, datasets, and learning rates is supported during reinforcement learning fine-tuning. The pipeline passes these parameters across SFT, reward model training, and RL stages to optimize model quality according to custom requirements.

What is the sequence of stages in a reinforcement learning from human feedback workflow?▼

A reinforcement learning from human feedback workflow sequentially executes supervised fine-tuning, reward model training, and reinforcement learning. This ensures checkpoint flow across stages, producing an aligned policy optimized via preference-based reward scores.

Does the Tinker API support training custom reward models from preference data?▼

The Tinker API supports training custom reward models from preference comparisons. It scores outputs during the reinforcement learning stage to maximize alignment with human preferences, optimizing the previously fine-tuned supervised policy.

How do I ensure checkpoint flow between SFT and RL stages in an RLHF pipeline?▼

Checkpoint flow between SFT and RL stages is managed automatically by the Tinker API orchestration. It ensures the fine-tuned policy and trained reward model are passed directly into the reinforcement learning loop without manual intervention.

Related Skills

Explore More Software Engineering Skills→
thinking-machines-labthinking-machines-lab
4.0k

dpo

Configure and run Direct Preference Optimization training with a model and preference dataset.

Official
Intermediate
lidge-junlidge-jun
5

hugging-face-model-trainer

Train and fine-tune language models via Hugging Face Jobs and TRL.

Community
Advanced
North-Shore-AINorth-Shore-AI

cns-tinker

Deploy Chiral Narrative Synthesis with a Tinker-based training and evaluation pipeline.

Official
Advanced
CHENyiru3CHENyiru3

trl

Train language models with reinforcement learning workflows including SFT, reward modeling, RLHF, and DPO.

Community
Advanced