ORION introduces a novel end-to-end autonomous driving framework that leverages vision-language models to generate driving actions, addressing the gap between semantic reasoning and vehicle control.

• Integrates vision-language understanding with action generation in a unified framework
• Incorporates a specialized Action Translator to convert language instructions into precise driving commands
• Achieves superior performance in closed-loop evaluations compared to existing methods
• Demonstrates enhanced causal reasoning capabilities in complex interactive scenarios

This breakthrough matters for engineering because it provides a more human-like approach to autonomous driving decisions, potentially improving safety and reliability in real-world applications where traditional systems struggle with complex reasoning.

ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation