Leveraging Textual Anatomical Knowledge for Class-Imbalanced Semi-Supervised Multi-Organ Segmentation
Journal:
arXiv
Published Date:
Jan 23, 2025
Abstract
Annotating 3D medical images demands substantial time and expertise, driving
the adoption of semi-supervised learning (SSL) for segmentation tasks. However,
the complex anatomical structures of organs often lead to significant class
imbalances, posing major challenges for deploying SSL in real-world scenarios.
Despite the availability of valuable prior information, such as inter-organ
relative positions and organ shape priors, existing SSL methods have yet to
fully leverage these insights. To address this gap, we propose a novel approach
that integrates textual anatomical knowledge (TAK) into the segmentation model.
Specifically, we use GPT-4o to generate textual descriptions of anatomical
priors, which are then encoded using a CLIP-based model. These encoded priors
are injected into the segmentation model as parameters of the segmentation
head. Additionally, contrastive learning is employed to enhance the alignment
between textual priors and visual features. Extensive experiments demonstrate
the superior performance of our method, significantly surpassing
state-of-the-art approaches. The source code will be available at:
https://github.com/Lunn88/TAK-Semi.
