Zero-Shot 3D Visual Grounding from Vision-Language Models
Journal:
arXiv
Published Date:
May 28, 2025
Abstract
3D Visual Grounding (3DVG) seeks to locate target objects in 3D scenes using
natural language descriptions, enabling downstream applications such as
augmented reality and robotics. Existing approaches typically rely on labeled
3D data and predefined categories, limiting scalability to open-world settings.
We present SeeGround, a zero-shot 3DVG framework that leverages 2D
Vision-Language Models (VLMs) to bypass the need for 3D-specific training. To
bridge the modality gap, we introduce a hybrid input format that pairs
query-aligned rendered views with spatially enriched textual descriptions. Our
framework incorporates two core components: a Perspective Adaptation Module
that dynamically selects optimal viewpoints based on the query, and a Fusion
Alignment Module that integrates visual and spatial signals to enhance
localization precision. Extensive evaluations on ScanRefer and Nr3D confirm
that SeeGround achieves substantial improvements over existing zero-shot
baselines -- outperforming them by 7.7% and 7.1%, respectively -- and even
rivals fully supervised alternatives, demonstrating strong generalization under
challenging conditions.
