Personalized federated learning for medical vision-language models via efficient fine-tuning and uncertainty-aware disentanglement.

Journal: Journal of biomedical informatics
Published Date:

Abstract

Privacy concerns and data heterogeneity remain critical barriers to advancing medical AI. While federated learning (FL) offers a decentralized solution, it often suffers from the Subspace Mismatch problem in clinical settings, where non-independent and identically distributed (non-IID) data drives clients towards divergent parameter manifolds. To address this, we propose a novel personalized federated learning (pFL) framework for vision-language medical tasks. Our approach employs a "separate-and-calibrate" strategy: First, we leverage Parameter-Efficient Fine-Tuning (PEFT) with a Dual-Subspace Disentanglement mechanism, splitting parameters into shared and personalized components to decouple general medical knowledge from site-specific characteristics. Second, to prevent negative transfer, we introduce Dynamic Likelihood-weighted Uncertainty Calibration (DLUC). This mechanism utilizes Dempster-Shafer theory to quantify epistemic uncertainty, acting as a reliability-aware gate that selectively aggregates compatible knowledge. Extensive experiments on Visual Question Answering, Question Generation, and Medical Report Generation demonstrate that our method significantly mitigates client drift and achieves superior personalization compared to state-of-the-art baselines.

Authors

Keywords

No keywords available for this article.