A Continual Learning-driven Model for Accurate and Generalizable Segmentation of Clinically Comprehensive and Fine-grained Whole-body Anatomies in CT
Journal:
arXiv
Published Date:
Mar 16, 2025
Abstract
Precision medicine in the quantitative management of chronic diseases and
oncology would be greatly improved if the Computed Tomography (CT) scan of any
patient could be segmented, parsed and analyzed in a precise and detailed way.
However, there is no such fully annotated CT dataset with all anatomies
delineated for training because of the exceptionally high manual cost, the need
for specialized clinical expertise, and the time required to finish the task.
To this end, we proposed a novel continual learning-driven CT model that can
segment complete anatomies presented using dozens of previously partially
labeled datasets, dynamically expanding its capacity to segment new ones
without compromising previously learned organ knowledge. Existing multi-dataset
approaches are not able to dynamically segment new anatomies without
catastrophic forgetting and would encounter optimization difficulty or
infeasibility when segmenting hundreds of anatomies across the whole range of
body regions. Our single unified CT segmentation model, CL-Net, can highly
accurately segment a clinically comprehensive set of 235 fine-grained
whole-body anatomies. Composed of a universal encoder, multiple optimized and
pruned decoders, CL-Net is developed using 13,952 CT scans from 20 public and
16 private high-quality partially labeled CT datasets of various vendors,
different contrast phases, and pathologies. Extensive evaluation demonstrates
that CL-Net consistently outperforms the upper limit of an ensemble of 36
specialist nnUNets trained per dataset with the complexity of 5% model size and
significantly surpasses the segmentation accuracy of recent leading Segment
Anything-style medical image foundation models by large margins. Our continual
learning-driven CL-Net model would lay a solid foundation to facilitate many
downstream tasks of oncology and chronic diseases using the most widely adopted
CT imaging.
