PRS-Med: Position Reasoning Segmentation with Vision-Language Model in Medical Imaging
Journal:
arXiv
Published Date:
May 17, 2025
Abstract
Recent advancements in prompt-based medical image segmentation have enabled
clinicians to identify tumors using simple input like bounding boxes or text
prompts. However, existing methods face challenges when doctors need to
interact through natural language or when position reasoning is required -
understanding spatial relationships between anatomical structures and
pathologies. We present PRS-Med, a framework that integrates vision-language
models with segmentation capabilities to generate both accurate segmentation
masks and corresponding spatial reasoning outputs. Additionally, we introduce
the MMRS dataset (Multimodal Medical in Positional Reasoning Segmentation),
which provides diverse, spatially-grounded question-answer pairs to address the
lack of position reasoning data in medical imaging. PRS-Med demonstrates
superior performance across six imaging modalities (CT, MRI, X-ray, ultrasound,
endoscopy, RGB), significantly outperforming state-of-the-art methods in both
segmentation accuracy and position reasoning. Our approach enables intuitive
doctor-system interaction through natural language, facilitating more efficient
diagnoses. Our dataset pipeline, model, and codebase will be released to foster
further research in spatially-aware multimodal reasoning for medical
applications.
