model-pruning

You can prune LLM weights without retraining by applying one-shot unstructured pruning methods like Wanda or SparseGPT. This approach uses activation-aware importance scoring to compress the model, reducing memory footprint and accelerating inference while preserving quality.

The best way to achieve hardware-friendly sparsity is using structured pruning and N:M sparsity patterns. These methods target accelerator-friendly inference speedups during LLM compression, ensuring the pruned model delivers actual hardware performance gains rather than just theoretical memory reductions.

Both Wanda and SparseGPT enable one-shot LLM pruning without retraining, using activation-aware importance scoring to determine which weights to remove. They differ in their specific scoring mechanisms and calibration approaches for measuring accuracy-speed tradeoffs during compression.

Yes, you can evaluate accuracy loss by comparing the pruned and original models. The workflow validates the sparsity level, measures the quality drop, assesses memory savings, and determines deployment readiness before exporting the compressed checkpoint for serving.

N:M sparsity requires compatible hardware accelerators to realize inference speedups; unsupported hardware yields no performance gain. Additionally, aggressive sparsity levels may degrade model accuracy, requiring careful evaluation of accuracy-speed tradeoffs during calibration and compression.

Activation-aware importance scoring is needed when performing one-shot unstructured or structured pruning to identify which weights to safely remove. It ensures the LLM compression process preserves model quality by evaluating weight significance based on actual activation patterns.