This research introduces a framework that bridges the gap between 2D vision-language models and 3D robotic task planning, enabling more precise robotic operations in dynamic environments.

Key Innovations:

• Automated prompt synthesis that improves 3D scene localization capabilities
• Supervised reasoning approach that enhances recognition accuracy and reliability
• Framework that addresses limitations in transferability for fine-grained robotic tasks
• Integration of vision-language models with 3D spatial understanding

Engineering Impact: This technology can transform factory automation by enabling robots to understand and interact with their environments more precisely, potentially reducing errors and increasing efficiency in manufacturing and industrial settings.

3D-Grounded Vision-Language Framework for Robotic Task Planning: Automated Prompt Synthesis and Supervised Reasoning