Steady Progress Beats Stagnation: Mutual Aid of Foundation and Conventional Models in Mixed Domain Semi-Supervised Medical Image Segmentation
Journal:
arXiv
Published Date:
Mar 21, 2025
Abstract
Large pretrained visual foundation models exhibit impressive general
capabilities. However, the extensive prior knowledge inherent in these models
can sometimes be a double-edged sword when adapting them to downstream tasks in
specific domains. In the context of semi-supervised medical image segmentation
with domain shift, foundation models like MedSAM tend to make overconfident
predictions, some of which are incorrect. The error accumulation hinders the
effective utilization of unlabeled data and limits further improvements. In
this paper, we introduce a Synergistic training framework for Foundation and
Conventional models (SynFoC) to address the issue. We observe that a
conventional model trained from scratch has the ability to correct the
high-confidence mispredictions of the foundation model, while the foundation
model can supervise it with high-quality pseudo-labels in the early training
stages. Furthermore, to enhance the collaborative training effectiveness of
both models and promote reliable convergence towards optimization, the
consensus-divergence consistency regularization is proposed. We demonstrate the
superiority of our method across four public multi-domain datasets. In
particular, our method improves the Dice score by 10.31\% on the Prostate
dataset. Our code is available at https://github.com/MQinghe/SynFoC .
