unsloth-advanced

What problem does it solve?

This Skill addresses the need for advanced techniques in model training, enabling deeper reasoning, preference alignment, and vision capabilities for unsloth models.

Core Features & Use Cases

• GRPO (Group Relative Policy Optimization): Enhances reasoning using custom Python reward functions without a value model, suitable for multi-GPU scenarios.
• DPO (Direct Preference Optimization): Aligns models with human preferences using paired preference data without a reward model.
• ORPO (Odds Ratio Preference Optimization): Combines SFT and preference alignment in a single training pass.
• Continued Pretraining (CPT): Teaches new knowledge or languages to models.
• Vision Model Fine-Tuning: Fine-tunes vision-language models for image understanding.
• Use Case: When building reasoning models, aligning with preferences, fine-tuning vision-language models, or teaching new knowledge/languages.

Quick Start

Train a GRPO model using the provided script and data.