FedSCA: Federated Tuning with Similarity-guided Collaborative Aggregation for Heterogeneous Medical Image Segmentation
Journal:
arXiv
Published Date:
Mar 19, 2025
Abstract
Transformer-based foundation models (FMs) have recently demonstrated
remarkable performance in medical image segmentation. However, scaling these
models is challenging due to the limited size of medical image datasets within
isolated hospitals, where data centralization is restricted due to privacy
concerns. These constraints, combined with the data-intensive nature of FMs,
hinder their broader application. Integrating federated learning (FL) with
foundation models (FLFM) fine-tuning offers a potential solution to these
challenges by enabling collaborative model training without data sharing, thus
allowing FMs to take advantage of a diverse pool of sensitive medical image
data across hospitals/clients. However, non-independent and identically
distributed (non-IID) data among clients, paired with computational and
communication constraints in federated environments, presents an additional
challenge that limits further performance improvements and remains inadequately
addressed in existing studies. In this work, we propose a novel FLFM
fine-tuning framework, \underline{\textbf{Fed}}erated tuning with
\underline{\textbf{S}}imilarity-guided \underline{\textbf{C}}ollaborative
\underline{\textbf{A}}ggregation (FedSCA), encompassing all phases of the FL
process. This includes (1) specially designed parameter-efficient fine-tuning
(PEFT) for local client training to enhance computational efficiency; (2)
partial low-level adapter transmission for communication efficiency; and (3)
similarity-guided collaborative aggregation (SGCA) on the server side to
address non-IID issues. Extensive experiments on three FL benchmarks for
medical image segmentation demonstrate the effectiveness of our proposed
FedSCA, establishing new SOTA performance.
