Smarter Autonomous Driving During Perception Failures

Smarter Autonomous Driving During Perception Failures

Using LLMs to Apply Human-like Commonsense in Critical Situations

This research introduces LLM-RCO, a framework that enables autonomous vehicles to make safer decisions when sensors or perception systems partially fail.

  • Integrates multimodal LLMs to apply commonsense reasoning during perception deficits instead of defaulting to immediate stops
  • Features modules for hazard inference, risk assessment, and safety-constrained decision-making
  • Proposes a more flexible approach for handling rare driving scenarios without disrupting traffic flow
  • Significantly enhances autonomous driving safety in compromised perception conditions

Important for Security: This framework addresses a critical vulnerability in autonomous systems by providing a fallback mechanism that mimics human-like reasoning rather than relying on pre-programmed emergency protocols.

Combating Partial Perception Deficit in Autonomous Driving with Multimodal LLM Commonsense

13 | 21