Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
-
Updated
Apr 18, 2024 - Python
Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
Add a description, image, and links to the kv-cache-compression topic page so that developers can more easily learn about it.
To associate your repository with the kv-cache-compression topic, visit your repo's landing page and select "manage topics."