3D-TAFS: Bridging Language and Robotic Action

3D-TAFS: Bridging Language and Robotic Action

A Training-Free Framework for 3D Affordance Understanding

3D-TAFS is a novel framework that translates natural language instructions into precise robotic actions without requiring additional training, enabling robots to better understand how to interact with objects in 3D space.

  • Introduces a training-free multimodal approach that connects language models with 3D vision networks
  • Developed IndoorAfford-Bench, a comprehensive benchmark with 9,248 images across 20 indoor scenes
  • Achieves superior performance in identifying functional parts of objects that humans can interact with
  • Enables zero-shot generalization to new objects and instructions

This research advances engineering capabilities for autonomous robots by allowing them to understand the "affordances" of objects—what actions can be performed with them—creating more intuitive human-robot interactions in real-world environments.

3D-TAFS: A Training-free Framework for 3D Affordance Segmentation

36 | 168