skills.rest
gagan114662
gagan114662COMMUNITY
Explore all skills by this author →
GitHub

model-pruning

Prune LLMs using Wanda and SparseGPT to reduce size and accelerate inference.

Updated Jan 30, 2026
Search Tags:#sparsity#inference acceleration#llm compression#model pruning#wanda#sparsegpt
One-click install
npx skills add https://github.com/gagan114662/content_books --skill model-pruning-gagan114662
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-pruning
Source: https://github.com/gagan114662/content_books/tree/main/AI-research-SKILLs/19-emerging-techniques/model-pruning
Command: npx skills add https://github.com/gagan114662/content_books --skill model-pruning-gagan114662

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, datasets, lm_eval, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of large, computationally expensive Large Language Models (LLMs) by enabling their compression to reduce size and accelerate inference times, making them more deployable on resource-constrained environments.

Core Features & Use Cases

  • Model Compression: Reduce LLM size significantly (e.g., 40-60%) with minimal accuracy loss (<1%).
  • Inference Acceleration: Achieve faster inference speeds (2-4x) by leveraging hardware-friendly sparsity.
  • Deployment on Edge Devices: Enable LLM deployment on devices with limited memory and processing power.
  • Use Case: You have a large Llama-2 7B model that needs to be deployed on a mobile device for real-time text generation. Use this Skill to prune the model to a smaller size and faster inference speed without a significant drop in performance.

Quick Start

Use the model-pruning skill to prune the 'meta-llama/Llama-2-7b-hf' model to 50% sparsity using the Wanda method.

Frequently Asked Questions about model-pruning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compress an LLM for faster inference without retraining?▼

LLM compression without retraining is achieved through one-shot pruning methods like Wanda and SparseGPT. These techniques reduce model size by 40-60% and accelerate inference speed by 2-4x with under 1% accuracy loss.

What is the difference between unstructured pruning and N:M sparsity for LLM compression?▼

Unstructured pruning removes individual weights across the entire model, while N:M sparsity enforces a fixed ratio of zero weights within contiguous blocks. Both methods are supported to create hardware-friendly sparse models for accelerated inference.

Can I deploy a pruned Llama-2 7B model on edge devices using PyTorch?▼

Yes, you can prune a Llama-2 7B model to 50% sparsity using PyTorch and Transformers to deploy on edge devices. This reduces memory requirements and enables real-time text generation on hardware with limited processing power.

Does magnitude pruning work well for achieving 50% sparsity in large language models?▼

Magnitude pruning is supported alongside Wanda and SparseGPT for achieving 50% sparsity in large language models. Wanda and SparseGPT generally preserve accuracy better during compression, while magnitude pruning offers a simpler baseline approach.

What are the limitations of using one-shot pruning methods for model compression?▼

One-shot pruning methods like Wanda and SparseGPT can achieve high sparsity with minimal accuracy loss, but extreme compression beyond 50-60% may cause significant performance degradation. Structured pruning may be needed for specific hardware acceleration requirements.

Related Skills

Explore More Software Engineering Skills→
ResearAIResearAI
3.3k

baseline

Standardize a reference object for quest evaluation with plan and checklist requirements.

Community
Advanced
ChimeraFoundationaChimeraFoundationa

evaluating-llms-harness

Benchmark LLMs across 60+ standardized tasks using lm_eval.

Community
Advanced
OpenLAIROpenLAIR
1.0k

model-pruning

Identify and apply model pruning to LLMs using Wanda or SparseGPT.

Official
Advanced
ihatesea69ihatesea69
3

awq-quantization

Quantize large language models to 4-bit weights using AWQ.

Community
Advanced