MedGround-R1: Advancing Medical Image Grounding via Spatial-Semantic Rewarded Group Relative Policy Optimization
Journal:
arXiv
Published Date:
Jul 1, 2025
Abstract
Medical Image Grounding (MIG), which involves localizing specific regions in
medical images based on textual descriptions, requires models to not only
perceive regions but also deduce spatial relationships of these regions.
Existing Vision-Language Models (VLMs) for MIG often rely on Supervised
Fine-Tuning (SFT) with large amounts of Chain-of-Thought (CoT) reasoning
annotations, which are expensive and time-consuming to acquire. Recently,
DeepSeek-R1 demonstrated that Large Language Models (LLMs) can acquire
reasoning abilities through Group Relative Policy Optimization (GRPO) without
requiring CoT annotations. In this paper, we adapt the GRPO reinforcement
learning framework to VLMs for Medical Image Grounding. We propose the
Spatial-Semantic Rewarded Group Relative Policy Optimization to train the model
without CoT reasoning annotations. Specifically, we introduce Spatial-Semantic
Rewards, which combine spatial accuracy reward and semantic consistency reward
to provide nuanced feedback for both spatially positive and negative
completions. Additionally, we propose to use the Chain-of-Box template, which
integrates visual information of referring bounding boxes into the
reasoning process, enabling the model to explicitly reason about spatial
regions during intermediate steps. Experiments on three datasets MS-CXR,
ChestX-ray8, and M3D-RefSeg demonstrate that our method achieves
state-of-the-art performance in Medical Image Grounding. Ablation studies
further validate the effectiveness of each component in our approach. Code,
checkpoints, and datasets are available at
https://github.com/bio-mlhui/MedGround-R1
