skills.rest
zechenzhangAGI
zechenzhangAGICOMMUNITY
Explore all skills by this author →
Website𝕏@ZechenZhang5GitHub

model-pruning

Prune large language models using one-shot methods like Wanda and SparseGPT.

11.5k|842|Updated Nov 3, 2025
Search Tags:#model compression#SparseGPT#LLM pruning#Wanda#model optimization#N:M sparsity#inference acceleration
One-click install
npx skills add https://github.com/zechenzhangAGI/AI-research-SKILLs --skill model-pruning
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-pruning
Source: https://github.com/zechenzhangAGI/AI-research-SKILLs/tree/main/19-emerging-techniques/model-pruning
Command: npx skills add https://github.com/zechenzhangAGI/AI-research-SKILLs --skill model-pruning

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, transformers, accelerate, and includes references (resource) components.

What problem does it solve?

This Skill tackles the challenges of deploying large language models, which are often slow and memory-intensive. It enables you to significantly reduce model size and accelerate inference, making LLMs more practical for edge devices and cost-effective serving.

Core Features & Use Cases

  • Model Compression: Reduce LLM size by 40-60% with minimal accuracy loss (typically <1%), making models lighter and easier to store.
  • Inference Acceleration: Achieve 2-4× speedup in inference on hardware accelerators by leveraging structured sparsity patterns like N:M pruning.
  • One-Shot Pruning: Compress models without the need for extensive and costly retraining, using efficient methods like Wanda and SparseGPT.
  • Efficient Deployment: Enable deployment on resource-constrained hardware (e.g., mobile, edge devices) and reduce the memory footprint for serving.
  • Use Case: Deploy a Llama-2-7b model on a mobile device or a low-cost GPU server, achieving faster response times and lower operational costs without sacrificing much performance.

Quick Start

Apply Wanda pruning to a Llama-2-7b-hf model to achieve 50% sparsity using a small calibration dataset, without any retraining.

Frequently Asked Questions about model-pruning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce LLM size and speed up inference without retraining?▼

Model pruning compresses large language models by removing less important weights, achieving 40-60% size reduction and 2-4× inference speedup. One-shot methods like Wanda and SparseGPT require only calibration data, no retraining.

What's the difference between unstructured and structured pruning for LLMs?▼

Unstructured pruning removes individual weights for maximum compression; structured pruning removes entire channels or layers for hardware efficiency. N:M sparsity is a structured pattern that balances both, enabling faster inference on accelerators.

Can I deploy a large language model on mobile or edge devices?▼

Model pruning enables mobile and edge deployment by reducing memory footprint and computation. Techniques like magnitude pruning and Wanda compress models to sizes practical for resource-constrained hardware while maintaining performance.

How do pruning methods like Wanda and SparseGPT work?▼

Wanda and SparseGPT are one-shot pruning methods that use activation statistics from calibration data to identify and remove unimportant weights, compressing models efficiently without expensive retraining cycles.

What calibration data do I need for model pruning?▼

Pruning methods collect activation statistics from calibration data to guide weight removal. A small representative dataset is sufficient for one-shot methods; the data should reflect your model's typical usage patterns.

Does PyTorch and Hugging Face Transformers support model pruning?▼

Yes, this Skill leverages torch, transformers, and accelerate to implement unstructured and structured pruning. These dependencies provide the foundation for magnitude pruning, N:M sparsity, Wanda, and SparseGPT methods.

Related Skills

Explore More Software Engineering Skills→
eliottbusinesseliottbusiness

huggingface-hub

Manage Hugging Face Hub models and datasets via the hf CLI.

Community
Intermediate
zechenzhangAGIzechenzhangAGI
11.5k

model-pruning

Prune large language models using one-shot methods like Wanda and SparseGPT.

Community
Intermediate
zechenzhangAGIzechenzhangAGI
11.5k

knowledge-distillation

Compress large language models by transferring teacher knowledge to smaller student models.

Community
Intermediate
informatico-madridinformatico-madrid
1

knowledge-distillation

Compress large language models into smaller student models using knowledge distillation.

Community
Advanced