LEO-MINI introduces a novel architecture that dramatically reduces visual token processing requirements while improving visual reasoning capabilities in multimodal LLMs.

Key Innovations:

• CoTR: Conditional Token Reduction technique that intelligently processes visual information
• MMoE: Mixture of Multi-Modal Experts architecture that enhances visual reasoning
• Efficiency gains without compromising - and even improving - reasoning capabilities
• Practical solution for reducing the computational burden of multimodal AI systems

This engineering breakthrough addresses a critical challenge in multimodal AI deployment: maintaining high-quality visual understanding while significantly reducing computational requirements, making advanced multimodal systems more accessible and deployable.

LEO-MINI: An Efficient Multimodal Large Language Model using Conditional Token Reduction and Mixture of Multi-Modal Experts