SimLingo introduces a novel approach to autonomous driving that achieves both high driving performance and comprehensive language understanding through vision-language alignment.

• Integrates vision-only perception with language models for more explainable autonomous driving
• Implements a closed-loop system where language and driving actions are naturally aligned
• Demonstrates improved driving performance while maintaining language understanding capabilities
• Provides a foundation for safer, more interpretable self-driving systems

This research represents a significant engineering advancement by creating driving systems that can not only navigate effectively but also understand and communicate about their environment and decisions, potentially improving both functionality and human trust in autonomous vehicles.

SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment