skills.rest
tianhao909
tianhao909COMMUNITY
Explore all skills by this author →
GitHub

model-pruning

Prune LLMs with Wanda and SparseGPT to reduce size and accelerate inference.

1|Updated Feb 21, 2026
Search Tags:#sparsity#inference acceleration#llm compression#model pruning#wanda#sparsegpt
One-click install
npx skills add https://github.com/tianhao909/AI-Research-SKILLs-cn --skill model-pruning-tianhao909
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-pruning
Source: https://github.com/tianhao909/AI-Research-SKILLs-cn/tree/main/19-emerging-techniques/model-pruning
Command: npx skills add https://github.com/tianhao909/AI-Research-SKILLs-cn --skill model-pruning-tianhao909

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, accelerate, datasets, lm_eval, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the challenge of large, computationally expensive Large Language Models (LLMs) by reducing their size and speeding up inference times through advanced pruning techniques.

Core Features & Use Cases

  • Model Compression: Reduce LLM size by 40-60% with minimal accuracy loss (<1%).
  • Inference Acceleration: Achieve 2-4x speedups using hardware-friendly sparsity.
  • Deployment on Constrained Hardware: Enable LLMs to run on devices with limited memory and processing power.
  • Use Case: Deploying a large language model on an edge device for real-time text generation or analysis where computational resources are scarce.

Quick Start

Use the model-pruning skill to prune the 'meta-llama/Llama-2-7b-hf' model to 50% sparsity using the Wanda method.

Frequently Asked Questions about model-pruning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compress a large language model for faster inference without extensive retraining?▼

You can compress a large language model for faster inference using pruning techniques like Wanda and SparseGPT. These methods reduce model size by 40-60% with minimal accuracy loss and without requiring extensive retraining.

What is the best way to achieve 50% sparsity on a Llama-2-7b model using Wanda?▼

The best way to achieve 50% sparsity on a Llama-2-7b model using Wanda is to apply the model-pruning workflow directly to the meta-llama/Llama-2-7b-hf weights, leveraging structured and unstructured sparsity for hardware-friendly acceleration.

Does model pruning with SparseGPT work for deploying LLMs on constrained edge hardware?▼

Model pruning with SparseGPT works for deploying LLMs on constrained edge hardware by achieving 2-4x inference speedups through hardware-friendly sparsity, enabling real-time text generation on devices with limited memory and processing power.

Can I use PyTorch and Hugging Face transformers to apply structured sparsity for LLM compression?▼

You can use PyTorch and Hugging Face transformers to apply structured sparsity for LLM compression. The workflow integrates these dependencies to reduce model size and accelerate processing while maintaining accuracy.

What are the limitations of using Wanda for model pruning compared to other LLM compression methods?▼

A limitation of using Wanda for model pruning is that while it achieves significant size reduction and speedups, there is still a minimal accuracy loss of less than 1% compared to the original dense model.

How much accuracy is lost when achieving 40-60% LLM compression through unstructured sparsity?▼

When achieving 40-60% LLM compression through unstructured sparsity, the accuracy loss is minimal, remaining under 1% compared to the original dense model baseline.

Related Skills

Explore More Software Engineering Skills→
Clay-HHKClay-HHK
2

model-pruning

Prune transformer models with Wanda, SparseGPT, and N:M sparsity patterns.

Community
Advanced
ChimeraFoundationaChimeraFoundationa

evaluating-llms-harness

Benchmark LLMs across 60+ standardized tasks using lm_eval.

Community
Advanced
Devsoul2026Devsoul2026
150

peft-fine-tuning

Fine-tune large language models with LoRA and QLoRA PEFT methods.

Community
Advanced
zechenzhangAGIzechenzhangAGI
11.5k

model-pruning

Prune large language models using one-shot methods like Wanda and SparseGPT.

Community
Intermediate