Multimodal Large Language Models (MLLMs) struggle with precise spatial understanding, limiting their effectiveness in collaborative robotics and real-world applications.

• Models face challenges in localizing and disambiguating objects in complex 3D environments
• Current MLLMs can generate realistic descriptions but lack precision in spatial instructions
• This research develops improved evaluation methods for spatial understanding capabilities
• Enhanced spatial reasoning is critical for safe human-AI collaboration in physical spaces

As MLLMs become more integrated with robotic systems, addressing these spatial understanding limitations will enable more reliable deployment in engineering and security applications where precise object identification is essential.

3D Spatial Understanding in MLLMs: Disambiguation and Evaluation