mergekit

What problem does it solve?

Mergekit solves the problem of combining multiple fine-tuned language model checkpoints into a single better generalist without needing additional training.

Core Features & Use Cases

• GPU-free model fusion: Create a merged model primarily via CPU workflows for many merge methods, avoiding costly retraining.
• Multiple merge strategies: Use SLERP, TIES, DARE, Task Arithmetic, Frankenmerge (layer stacking), and Evolutionary merge to control how capabilities combine.
• Practical outcomes: Combine coding/math/reasoning specialists to reduce catastrophic forgetting and improve breadth compared to selecting a single checkpoint.
• Use Case: You have separate LoRA/finetune outputs for math and coding on the same base architecture; Mergekit merges them into one checkpoint that balances both abilities.

Quick Start

Ask the AI to merge your checkpoints into an output directory by running mergekit on a SLERP YAML config file (e.g., mergekit-yaml slerp_merge.yaml ./merged-model --copy-tokenizer).