RestoreKV

Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

Changwoo Baek¹  ·  Seungjun Shin^2†  ·  Kyeongbo Kong^1†
1Pusan National University  ·  ²Sookmyung Women's University

---

🏆 #1 on the KVPress Leaderboard.

RestoreKV complements selection-based query-agnostic KV cache eviction with learned restoration under the same total KV budget. After context prefill, a few restore tokens attend to the full KV cache in a single LoRA-adapted pass, generating a compact, context-conditioned restore cache. The base importance scorer and eviction rule remain unchanged, and the adapters are disabled for all subsequent queries and decoding. RestoreKV is trained through parameter-efficient self-distillation from the frozen full-cache model, optimizing only 0.4% of the parameters and requiring no task-specific tuning.

• Improves 59 of 60 paired, budget-matched settings across five base eviction methods (Qwen3-4B).
• At a 5% budget, raises KVzip from 38.2 → 73.2 on RULER-4K.
• Applied to KVzip+, reaches 86.4 RULER accuracy at 16× compression on the KVPress Benchmark, while adding <0.5% one-time cache-construction overhead in a 32K-context evaluation.

🔗 Resources

📄 Paper	arXiv:2608.01247
🌎 Project page	paper.pnu-cvsp.com/RestoreKV
⚙️ Inference code	restorekv_press.py in NVIDIA/KVPress
🤗 Weights	huggingface.co/collections/higokri/restorekv
🏆 Leaderboard	KVPress Leaderboard — 1st place

🚀 Usage

Inference is available through NVIDIA KVPress via RestoreKVPress, with pretrained restore adapters hosted on the Hugging Face collection.

from kvpress import RestoreKVPress

# Wrap any base eviction press with a learned, budget-matched restore pass.
press = RestoreKVPress(...)

See kvpress/presses/restorekv_press.py for the full inference implementation and arguments.

🗺️ Release Plan

• Inference code (integrated into NVIDIA/KVPress)
• Pretrained restore adapters (Hugging Face)
• Full training code — coming soon

Note. The full training / self-distillation code is currently being prepared and will be released here. In the meantime, inference is fully available through KVPress with the released weights.

📚 Citation

@article{baek2026restorekv,
  title   = {RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction},
  author  = {Baek, Changwoo and Shin, Seungjun and Kong, Kyeongbo},
  journal = {arXiv preprint arXiv:2608.01247},
  year    = {2026}
}

🙏 Acknowledgements

RestoreKV builds on KVzip for query-agnostic context-reconstruction eviction — we thank the authors for their great work — and its inference is implemented on top of NVIDIA KVPress. RestoreKV is part of the BTS — Busan Token-pruning Series from the PNU-CVSP lab.