
Smarter Autonomous Driving During Perception Failures
Using LLMs to Apply Human-like Commonsense in Critical Situations
This research introduces LLM-RCO, a framework that enables autonomous vehicles to make safer decisions when sensors or perception systems partially fail.
- Integrates multimodal LLMs to apply commonsense reasoning during perception deficits instead of defaulting to immediate stops
- Features modules for hazard inference, risk assessment, and safety-constrained decision-making
- Proposes a more flexible approach for handling rare driving scenarios without disrupting traffic flow
- Significantly enhances autonomous driving safety in compromised perception conditions
Important for Security: This framework addresses a critical vulnerability in autonomous systems by providing a fallback mechanism that mimics human-like reasoning rather than relying on pre-programmed emergency protocols.
Combating Partial Perception Deficit in Autonomous Driving with Multimodal LLM Commonsense