skills.rest
qcmuu
qcmuuCOMMUNITY
Explore all skills by this author →
GitHub

model-pruning

Prune LLMs using Wanda, SparseGPT, structured pruning, and N:M sparsity.

Updated May 2, 2026
Search Tags:#model compression#inference acceleration#wanda#sparsegpt#llm pruning#n:m sparsity#torch transformers
One-click install
npx skills add https://github.com/qcmuu/AI-Research-Skills --skill model-pruning-qcmuu
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-pruning
Source: https://github.com/qcmuu/AI-Research-Skills/tree/main/19-emerging-techniques/model-pruning
Command: npx skills add https://github.com/qcmuu/AI-Research-Skills --skill model-pruning-qcmuu

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, and includes references (resource) components.

What problem does it solve?

Model pruning reduces LLM compute and memory usage while preserving accuracy by removing redundant parameters using pruning criteria such as Wanda and SparseGPT.

Core Features & Use Cases

  • Untrained (one-shot) pruning: Compress models without retraining by using calibration activations or second-order information.
  • Hardware-aware sparsity: Apply N:M sparsity (e.g., 2:4) to unlock accelerator-friendly speedups on supported GPUs.
  • Structured vs unstructured options: Choose fine-grained sparsity for quality or structured sparsity for more reliable hardware execution.

Use cases: Deploying LLMs on constrained hardware, accelerating inference in production, reducing model footprint for edge/mobile deployment, and enabling faster serving without costly retraining.

Quick Start

Tell your AI agent to prune a Hugging Face Llama checkpoint to 50% sparsity using Wanda with a small calibration dataset, then save the resulting model for evaluation and deployment.

Frequently Asked Questions about model-pruning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prune a Hugging Face LLM to reduce its size without retraining?▼

You can apply one-shot pruning methods like Wanda or SparseGPT to compress Hugging Face models without retraining. These techniques use small calibration datasets or second-order information to remove redundant parameters while preserving model accuracy.

What is the difference between structured pruning and N:M sparsity for inference acceleration?▼

Structured pruning removes entire model structures for reliable hardware execution, while N:M sparsity applies fine-grained patterns like 2:4 to unlock specific accelerator-friendly GPU speedups. N:M offers hardware compatibility, whereas structured pruning provides broader execution reliability.

Does Wanda pruning require calibration data to compress a model?▼

Yes, Wanda pruning requires calibration activations to determine which parameters are redundant. This calibration data allows the pruning process to accurately evaluate parameter importance and remove weights without causing significant accuracy degradation.

Can I use N:M sparsity patterns with torch transformers for faster GPU inference?▼

Yes, you can apply N:M sparsity patterns like 2:4 to torch transformers models. This hardware-aware sparsity unlocks accelerator-friendly speedups on supported GPUs, enabling faster inference compared to unstructured pruning approaches.

When should I use unstructured sparsity instead of structured pruning for LLM compression?▼

Use unstructured sparsity when you need fine-grained sparsity to maximize model quality, and use structured pruning when you need more reliable hardware execution. Unstructured pruning removes individual weights, while structured pruning eliminates larger parameter blocks.

What are the limitations of using SparseGPT for model compression?▼

SparseGPT requires second-order statistics and correct sparsity masking to function properly. While it avoids costly retraining, improper calibration or masking can lead to significant accuracy drops, making it less suitable for models without adequate calibration data.

Related Skills

Explore More Education & Research Skills→
Clay-HHKClay-HHK
2

model-pruning

Prune transformer models with Wanda, SparseGPT, and N:M sparsity patterns.

Community
Advanced
tuliopc23tuliopc23

coreml-diag

Diagnose CoreML model loading, performance, and conversion issues on Apple devices.

Community
Advanced
KappTech88KappTech88

awq-quantization

Quantize large language model weights to 4-bit with activation-aware AWQ.

Community
Advanced
informatico-madridinformatico-madrid
1

knowledge-distillation

Compress large language models into smaller student models using knowledge distillation.

Community
Advanced