Brain Foundation Models with Hypergraph Dynamic Adapter for Brain Disease Analysis
Journal:
arXiv
Published Date:
May 1, 2025
Abstract
Brain diseases, such as Alzheimer's disease and brain tumors, present
profound challenges due to their complexity and societal impact. Recent
advancements in brain foundation models have shown significant promise in
addressing a range of brain-related tasks. However, current brain foundation
models are limited by task and data homogeneity, restricted generalization
beyond segmentation or classification, and inefficient adaptation to diverse
clinical tasks. In this work, we propose SAM-Brain3D, a brain-specific
foundation model trained on over 66,000 brain image-label pairs across 14 MRI
sub-modalities, and Hypergraph Dynamic Adapter (HyDA), a lightweight adapter
for efficient and effective downstream adaptation. SAM-Brain3D captures
detailed brain-specific anatomical and modality priors for segmenting diverse
brain targets and broader downstream tasks. HyDA leverages hypergraphs to fuse
complementary multi-modal data and dynamically generate patient-specific
convolutional kernels for multi-scale feature fusion and personalized
patient-wise adaptation. Together, our framework excels across a broad spectrum
of brain disease segmentation and classification tasks. Extensive experiments
demonstrate that our method consistently outperforms existing state-of-the-art
approaches, offering a new paradigm for brain disease analysis through
multi-modal, multi-scale, and dynamic foundation modeling.
