skills.rest
thistleknot
thistleknotCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

model-size-reduction

Reduce Hugging Face checkpoint size via dtype casting, layer dropping, and LoRA extraction.

Updated Apr 20, 2026
Search Tags:#model compression#Hugging Face#checkpoint slimming#LoRA extraction#delta sparsification#DARE TIES DELLA#perplexity validation
One-click install
npx skills add https://github.com/thistleknot/skills --skill model-size-reduction
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-size-reduction
Source: https://github.com/thistleknot/skills/tree/main/model-size-reduction
Command: npx skills add https://github.com/thistleknot/skills --skill model-size-reduction

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Model size reduction solves the problem of deploying or sharing oversized Hugging Face checkpoints when tooling like mergekit cannot handle the architecture or when you need a smaller, more portable artifact.

Core Features & Use Cases

  • Architecture-agnostic slimming: Recasts dtype, drops transformer blocks via state_dict key surgery, extracts LoRA from finetuned-minus-base, and performs delta sparsification using DARE/TIES/DELLA.
  • Checkpointed multi-stage pipelines: Runs reduction as a resumable series of full model directories so interrupted jobs can continue.
  • Validation guardrails: Measures reduction aggressiveness with perplexity (relative delta) and provides failure boundaries like unrecognized layer-key patterns or missing safetensors.

Quick Start

Use this skill to reduce a Hugging Face checkpoint by first sparsifying the fine-tune delta (DARE/TIES/DELLA), then optionally dropping layers, casting to fp16 or bf16, and finally validating perplexity before publishing.

Frequently Asked Questions about model-size-reduction

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I shrink a Hugging Face checkpoint when mergekit fails architecture detection?▼

Reduce Hugging Face checkpoint size by applying architecture-agnostic dtype casting, layer dropping via state_dict key surgery, and delta sparsification to create portable artifacts without relying on mergekit.

What is delta sparsification for model compression?▼

Delta sparsification compresses models by applying DARE, TIES, or DELLA techniques to the difference between fine-tuned and base checkpoints, pruning redundant weights while preserving task performance.

How do I extract LoRA adapters from a fine-tuned Hugging Face model?▼

Extract LoRA adapters by computing the difference between the fine-tuned checkpoint and the matching base checkpoint, isolating the task-specific delta weights for portable deployment.

Can I validate perplexity after dropping transformer layers from a checkpoint?▼

Yes, validate perplexity by measuring the relative delta between the reduced and source checkpoints, ensuring the reduction aggressiveness does not cause excessive quality loss before publishing.

What are the limitations of architecture-agnostic checkpoint slimming?▼

Architecture-agnostic checkpoint slimming requires state_dict keys to follow numeric transformer layer patterns, needs matching base checkpoints for adapter or delta methods, and fails on unrecognized layer-key patterns or missing safetensors.

Do I need a matching base checkpoint to sparsify a fine-tuned model delta?▼

Yes, delta sparsification using DARE, TIES, or DELLA requires access to both the fine-tuned checkpoint weights and the matching base checkpoint to calculate and prune the difference accurately.

Related Skills

Explore More Software Engineering Skills→
gar-aigar-ai
1

mithril-checkpoint-agent

Compress PyTorch model checkpoints with mithril-checkpoint for lossless reduction.

Official
Advanced
oabdelmaksoudoabdelmaksoud
4

hugging-face-paper-publisher

Index, publish, and manage academic papers on the Hugging Face Hub.

Community
Advanced
ricablericable

model-merging

Merges multiple fine-tuned language models into one checkpoint without retraining.

Community
Advanced
thistleknotthistleknot

mergekit — Skill

Merge multiple LLM checkpoints and LoRA deltas into a single .safetensors artifact.

Community
Advanced