skills.rest
JoaquinCampo
JoaquinCampoCOMMUNITY
Explore all skills by this author ā†’
GitHub

kvpress

Compress HuggingFace transformer model KV-cache during generation with press strategies.

3|Updated Dec 22, 2025
Search Tags:#pytorch#huggingface#memory-optimization#kv-cache#llm-generation#attention-pruning#kvpress
One-click install
npx skills add https://github.com/JoaquinCampo/Skills --skill kvpress
Or copy as Structured Prompt for Agentā–¼
Please help me install this Agent Skill.
Skill: kvpress
Source: https://github.com/JoaquinCampo/Skills/tree/main/kvpress
Command: npx skills add https://github.com/JoaquinCampo/Skills --skill kvpress

SYSTEM DOCUMENTATION & REQUIREMENTS

šŸ’” This Skill includes references (resource) components.

What problem does it solve?

kvpress compresses the KV-cache of HuggingFace transformers models during generation, reducing memory usage at the cost of potential quality trade-offs.

Core Features & Use Cases

  • Supports multiple press strategies (StreamingLLMPress, SnapKVPress, KnormPress, etc.) to prune the KV-cache during prefill.
  • Compatible with wrappers and per-layer configurations for flexible deployment across models.
  • Use cases include running large LLMs on limited GPUs, streaming generation, and experimenting with memory-quality trade-offs.

Quick Start

Wrap your model with a press, e.g., StreamingLLMPress(compression_ratio=0.5), then run model.generate() to enable compressed KV-cache during generation.

Frequently Asked Questions about kvpress

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I reduce KV-cache memory usage for HuggingFace transformer models during generation?ā–¼

You can reduce KV-cache memory by applying press strategies like StreamingLLMPress or SnapKVPress to compress the cache during the prefill stage. This lowers peak memory usage at the cost of potential quality trade-offs.

Does kvpress work with any PyTorch-based HuggingFace model?ā–¼

kvpress works with PyTorch-based HuggingFace models that expose the model.model.layers[].self_attn structure. It also supports wrappers and per-layer configurations to suit different model architectures.

What are the different KV-cache compression strategies available for LLM generation?ā–¼

Available KV-cache compression strategies include StreamingLLMPress, SnapKVPress, KnormPress, TOVAPress, and KVzipPress. These presses prune the KV-cache during prefill to help run large LLMs on limited GPUs.

How do I enable KV-cache compression when running LLM generation?ā–¼

To enable KV-cache compression, wrap your HuggingFace model with a press such as StreamingLLMPress(compression_ratio=0.5), then call model.generate() to run generation with the compressed cache.

What are the limitations of pruning the KV-cache during LLM prefill?ā–¼

Pruning the KV-cache during prefill reduces peak memory usage but introduces a potential quality trade-off. It is best suited for running large LLMs on limited GPUs, streaming generation, and experimenting with memory-quality balances.

Can I configure different KV-cache compression settings for individual transformer layers?ā–¼

Yes, kvpress supports per-layer configurations and wrappers, allowing you to apply flexible deployment settings across different model architectures during generation.

Related Skills

Explore More Software Engineering Skillsā†’
nelohenriqnelohenriq
1

optimizing-attention-flash

Integrate Flash Attention variants into PyTorch transformer models for NVIDIA GPUs.

Community
Advanced
hung-phanhung-phan
1

attention

Analyze attention mechanism performance, memory usage, and implementation trade-offs for transformer architectures.

Community
Advanced
KappTech88KappTech88

optimizing-attention-flash

Replace slow transformer attention with Flash Attention on NVIDIA GPUs using PyTorch.

Community
Advanced
Orchestra-ResearchOrchestra-Research
11.5k

mamba-architecture

Implement Mamba state-space models with linear complexity for long sequences.

Official
Advanced