Zero-Shot Gaze-based Volumetric Medical Image Segmentation
Journal:
arXiv
Published Date:
May 21, 2025
Abstract
Accurate segmentation of anatomical structures in volumetric medical images
is crucial for clinical applications, including disease monitoring and cancer
treatment planning. Contemporary interactive segmentation models, such as
Segment Anything Model 2 (SAM-2) and its medical variant (MedSAM-2), rely on
manually provided prompts like bounding boxes and mouse clicks. In this study,
we introduce eye gaze as a novel informational modality for interactive
segmentation, marking the application of eye-tracking for 3D medical image
segmentation. We evaluate the performance of using gaze-based prompts with
SAM-2 and MedSAM-2 using both synthetic and real gaze data. Compared to
bounding boxes, gaze-based prompts offer a time-efficient interaction approach
with slightly lower segmentation quality. Our findings highlight the potential
of using gaze as a complementary input modality for interactive 3D medical
image segmentation.
