CITER introduces a novel framework that dynamically routes tokens between large and small language models, significantly improving inference efficiency without sacrificing quality.

• Implements token-level routing to assign complex tokens to large models and simpler ones to small models
• Reduces computational costs for resource-constrained applications
• Achieves efficient collaboration between models of different sizes
• Maintains performance quality while improving inference speed

This engineering breakthrough enables more practical deployment of powerful language models in environments with limited computational resources, making advanced AI capabilities more accessible.

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing