skills.rest
thinking-machines-lab
thinking-machines-labOFFICIAL
Explore all skills by this author →
WebsiteGitHub

hyperparams

Calculate learning rate, LoRA rank, batch size, and schedule for fine-tuning language models.

4.0k|507|Updated Jul 14, 2025
Search Tags:#lora#hyperparameters#model-training#finetuning#learning-rate#batch-size#lr-schedule
One-click install
npx skills add https://github.com/thinking-machines-lab/tinker-cookbook --skill hyperparams
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hyperparams
Source: https://github.com/thinking-machines-lab/tinker-cookbook/tree/main/.claude/skills/hyperparams
Command: npx skills add https://github.com/thinking-machines-lab/tinker-cookbook --skill hyperparams

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Helps ML engineers quickly select validated hyperparameters for fine-tuning language models, reducing trial-and-error and speeding up experiments.

Core Features & Use Cases

  • Formula-based learning rate and LoRA calculations for common model families.
  • Guided recommendations for batch sizes and schedules across supervised, RL, DPO, and distillation workflows.
  • Scenario-driven guidance tailored to model size, architecture, and task requirements.

Quick Start

Use the hyperparams skill to derive learning rate, LoRA rank, batch size, and schedule recommendations for your model and training task.

Frequently Asked Questions about hyperparams

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate the learning rate for fine-tuning a language model?▼

LoRA rank is derived through model-specific calculations that factor in architecture and task requirements. The skill provides ready-to-use LoRA rank recommendations to reduce trial-and-error during language model fine-tuning.

What is the best way to choose batch size and schedule for DPO training?▼

Batch size and schedule selection for DPO is guided by scenario-driven recommendations tailored to model size and task. This approach produces validated hyperparameter values across DPO and distillation workflows.

Can I use this for hyperparameter tuning across supervised learning and reinforcement learning?▼

Yes, hyperparameter tuning applies to supervised learning, reinforcement learning, DPO, and distillation workflows. Guided recommendations adjust learning rate, LoRA rank, and batch size based on the specific training method.

Do I need to manually test different hyperparameters when fine-tuning large language models?▼

Manual trial-and-error is minimized by using validated, formula-based hyperparameter calculations. Scenario-driven guidance tailors learning rate and batch size recommendations to model size and architecture for faster experiments.

Related Skills

Explore More Software Engineering Skills→
camelranchentertainmentcamelranchentertainment
1

fine-tuning-expert

Fine-tune language models with LoRA/QLoRA adapters on JSONL datasets.

Community
Advanced
automatedigitalautomatedigital
1

unsloth

Fine-tune large language models with LoRA and QLoRA optimization.

Official
Intermediate
gagan114662gagan114662

tensorboard

Visualize machine learning training metrics and model graphs with TensorBoard.

Community
Intermediate
h4vzzh4vzz
33

hyperparameter-tuning

Explore hyperparameter configurations using grid, random, Bayesian, or Hyperband strategies.

Community
Advanced