This research investigates how KV cache compression affects large language models' fundamental capabilities, introducing a novel compression approach that maintains performance.

• Revealed significant performance degradation in core LLM abilities when using existing compression methods
• Demonstrated that compression methods optimized for long-context tasks often harm fundamental capabilities
• Introduced ShotKV, a new compression approach that better preserves model intelligence
• Established a comprehensive evaluation framework for assessing compression impact across diverse tasks

For AI engineers, this work provides crucial guidance on deploying efficient LLMs without compromising their essential capabilities—enabling both computational efficiency and preserved model quality.

Can LLMs Maintain Fundamental Abilities under KV Cache Compression?