model-merging

Combine multiple fine-tuned language models using SLERP, TIES-Merging, DARE, and Task Arithmetic.

1|Updated Feb 10, 2026

Search Tags:#llm #fine-tuning #ai models #model merging #mergekit #model fusion

Ready to run

npx skills add https://github.com/informatico-madrid/Architect-Expert-Gap-Forge --skill model-merging-informatico-madrid

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: model-merging
Source: https://github.com/informatico-madrid/Architect-Expert-Gap-Forge/tree/main/.github/skills/model-merging
Command: npx skills add https://github.com/informatico-madrid/Architect-Expert-Gap-Forge --skill model-merging-informatico-madrid

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mergekit, transformers, torch, and includes references (resource) components.

What problem does it solve?

This Skill addresses the challenge of creating more capable AI models by combining the strengths of multiple pre-trained models without the need for computationally expensive retraining.

Core Features & Use Cases

Model Fusion: Blend capabilities from different fine-tuned models (e.g., math, coding, chat) into a single, more versatile model.
Efficiency: Achieve enhanced performance and specialized skills with significantly reduced computational cost and time compared to training from scratch.
Use Case: Merge a model strong in mathematical reasoning with one proficient in code generation to create a single AI assistant that can solve complex math problems and write Python code.

Quick Start

Use the model-merging skill to merge the 'mistralai/Mistral-7B-v0.1' and 'teknium/OpenHermes-2.5-Mistral-7B' models using a linear merge method with equal weights.

Frequently Asked Questions about model-merging

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I combine fine-tuned LLMs without retraining?▼

Combine fine-tuned LLMs without retraining by applying model merging techniques like SLERP, TIES-Merging, DARE, and Task Arithmetic to blend their domain expertise into a single model. This avoids the computational cost of training from scratch.

What is the best way to fuse math and coding capabilities from different models?▼

The best way to fuse math and coding capabilities is using model fusion to blend a math-reasoning model with a code-generation model. This creates a versatile AI assistant specialized across both domains without needing further training.

Can I use mergekit to create Mixture of Experts architectures from existing models?▼

Yes, you can use mergekit to create Mixture of Experts architectures from existing models. The model merging skill supports advanced configurations for layer-wise merging and MoE architectures to facilitate specialized AI agents.

Does model fusion work with PyTorch and Hugging Face transformers?▼

Model fusion works with PyTorch and Hugging Face transformers, as both are required dependencies alongside mergekit. This allows you to seamlessly load and merge transformer-based language models.

What are the limitations of using Task Arithmetic for model merging?▼

Task Arithmetic for model merging blends models without retraining, meaning it inherits limitations from the base models and cannot learn entirely new capabilities. It is best suited for blending existing domain-specific expertise rather than acquiring novel knowledge.