kvpress

Compress HuggingFace transformer model KV-cache during generation with press strategies.

3|Updated Dec 22, 2025

Search Tags:#pytorch #huggingface #memory-optimization #kv-cache #llm-generation #attention-pruning #kvpress

Ready to run

npx skills add https://github.com/JoaquinCampo/Skills --skill kvpress

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: kvpress
Source: https://github.com/JoaquinCampo/Skills/tree/main/kvpress
Command: npx skills add https://github.com/JoaquinCampo/Skills --skill kvpress

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

kvpress compresses the KV-cache of HuggingFace transformers models during generation, reducing memory usage at the cost of potential quality trade-offs.

Core Features & Use Cases

Supports multiple press strategies (StreamingLLMPress, SnapKVPress, KnormPress, etc.) to prune the KV-cache during prefill.
Compatible with wrappers and per-layer configurations for flexible deployment across models.
Use cases include running large LLMs on limited GPUs, streaming generation, and experimenting with memory-quality trade-offs.

Quick Start

Wrap your model with a press, e.g., StreamingLLMPress(compression_ratio=0.5), then run model.generate() to enable compressed KV-cache during generation.

Frequently Asked Questions about kvpress

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I reduce KV-cache memory usage for HuggingFace transformer models during generation?▼

You can reduce KV-cache memory by applying press strategies like StreamingLLMPress or SnapKVPress to compress the cache during the prefill stage. This lowers peak memory usage at the cost of potential quality trade-offs.

Does kvpress work with any PyTorch-based HuggingFace model?▼

kvpress works with PyTorch-based HuggingFace models that expose the model.model.layers[].self_attn structure. It also supports wrappers and per-layer configurations to suit different model architectures.

What are the different KV-cache compression strategies available for LLM generation?▼

Available KV-cache compression strategies include StreamingLLMPress, SnapKVPress, KnormPress, TOVAPress, and KVzipPress. These presses prune the KV-cache during prefill to help run large LLMs on limited GPUs.

How do I enable KV-cache compression when running LLM generation?▼

To enable KV-cache compression, wrap your HuggingFace model with a press such as StreamingLLMPress(compression_ratio=0.5), then call model.generate() to run generation with the compressed cache.

What are the limitations of pruning the KV-cache during LLM prefill?▼

Pruning the KV-cache during prefill reduces peak memory usage but introduces a potential quality trade-off. It is best suited for running large LLMs on limited GPUs, streaming generation, and experimenting with memory-quality balances.

Can I configure different KV-cache compression settings for individual transformer layers?▼

Yes, kvpress supports per-layer configurations and wrappers, allowing you to apply flexible deployment settings across different model architectures during generation.