skills.rest
Orchestra-Research
Orchestra-ResearchOFFICIAL
Explore all skills by this author →
GitHub

model-pruning

Prune LLMs using unstructured, structured, N:M, and one-shot methods.

11.5k|842|Updated Nov 3, 2025
Search Tags:#sparsity#inference acceleration#llm compression#model pruning#wanda#sparsegpt
One-click install
npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill model-pruning-orchestra-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-pruning
Source: https://github.com/Orchestra-Research/AI-Research-SKILLs/tree/main/19-emerging-techniques/model-pruning
Command: npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill model-pruning-orchestra-research

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, accelerate, datasets, sparsegpt, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of large, computationally expensive Large Language Models (LLMs) by enabling their compression and accelerating inference speeds.

Core Features & Use Cases

  • Model Compression: Reduce LLM size significantly (e.g., 40-60%) with minimal accuracy loss (<1%).
  • Inference Acceleration: Achieve faster inference times (2-4x speedup) by utilizing hardware-friendly sparsity.
  • Deployment on Constrained Hardware: Enable LLM deployment on devices with limited memory and processing power.
  • Use Case: Deploying a large language model on an edge device for real-time natural language understanding tasks where computational resources are scarce.

Quick Start

Use the model-pruning skill to prune the 'meta-llama/Llama-2-7b-hf' model to 50% sparsity using the Wanda method.

Frequently Asked Questions about model-pruning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prune an LLM for faster inference on constrained hardware?▼

LLM pruning reduces model size and accelerates inference by applying techniques like unstructured pruning or N:M sparsity, enabling deployment on resource-constrained devices with 2-4x speedup.

What is the difference between Wanda and SparseGPT for model compression?▼

Wanda and SparseGPT are one-shot pruning methods supported for LLM compression. Both reduce model size significantly with minimal accuracy loss, but utilize different underlying mechanisms to achieve sparsity without retraining.

Can I achieve 50% sparsity on a Llama model without losing accuracy?▼

Yes, you can prune models like Llama-2-7b to 50% sparsity using Wanda or SparseGPT. These methods achieve 40-60% size reduction while maintaining accuracy loss below 1%.

What Python dependencies do I need to run structured pruning on transformers?▼

You need PyTorch, transformers, accelerate, datasets, and sparsegpt libraries installed to execute structured and unstructured pruning scripts on Hugging Face models.

Does magnitude pruning work for accelerating inference on edge devices?▼

Magnitude pruning helps reduce LLM size for edge deployment, but structured pruning or N:M sparsity provide better hardware-friendly inference acceleration for resource-constrained devices.

Why does unstructured pruning not speed up inference on my hardware?▼

Unstructured pruning reduces model size but lacks hardware-friendly sparsity patterns. For actual inference acceleration, use structured pruning or N:M sparsity to achieve 2-4x speedup on accelerators.

Related Skills

Explore More Software Engineering Skills→
Clay-HHKClay-HHK
2

model-pruning

Prune transformer models with Wanda, SparseGPT, and N:M sparsity patterns.

Community
Advanced
NVIDIANVIDIA
13.2k

trt-onnx-quickstart

Convert ONNX models to TensorRT engines and verify numerical parity.

Official
Advanced
ricablericable

model-pruning

Reduces LLM size via Wanda or SparseGPT pruning workflows.

Community
Advanced
ovachieverovachiever
52

gptq

Compress large language models to 4-bit precision with GPTQ.

Community
Advanced