This research introduces novel decoding methods that generate multiple high-quality tokens simultaneously, significantly improving LLM inference efficiency.

• Multi-Token Joint Decoding (MTJD) uses an auxiliary model to produce coherent multi-token outputs
• Optimized processing reduces inference latency by up to 2.06× while maintaining output quality
• Multi-Token Assisted Decoding (MTAD) further enhances performance through strategic auxiliary model integration
• Achieves substantial energy savings through reduced computational demands

This engineering breakthrough addresses a critical bottleneck in LLM deployment, enabling more cost-effective and sustainable AI applications at scale.

Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference