MILLION presents a breakthrough quantization technique that enables efficient processing of extremely long contexts (up to 1M tokens) in large language models.

• Introduces outlier-immunized KV product quantization that achieves up to 8x memory reduction with minimal quality loss
• Delivers 4x inference speedup through specialized GPU kernels and memory optimization
• Maintains model quality by specifically addressing the challenge of outliers in KV cache quantization
• Demonstrates practical deployment capabilities across multiple popular LLM architectures

This research is particularly valuable for engineering teams building applications requiring long document processing, complex reasoning, or extended conversations with memory constraints.

MILLION: Mastering Long-Context LLM Inference Via Outlier-Immunized KV Product Quantization