Open Vocabulary 3D Scene Understanding

Open Vocabulary 3D Scene Understanding

Advancing 3D segmentation with CLIP and superpoints

SPNeRF introduces a novel approach for segmenting 3D scenes using CLIP embeddings combined with geometric primitive representations.

  • Leverages superpoints (geometric primitives) to enhance CLIP's capabilities for 3D scene understanding
  • Enables open vocabulary segmentation beyond predefined classes for zero-shot 3D understanding
  • Overcomes CLIP's limitations in capturing geometric details necessary for accurate 3D segmentation
  • Presents a more efficient approach than methods requiring additional segmentation models

This research matters for engineering applications by bridging the gap between 2D vision-language models and 3D scene understanding, enabling more flexible and powerful 3D modeling systems without extensive labeled training data.

SPNeRF: Open Vocabulary 3D Neural Scene Segmentation with Superpoints

49 | 66