Unified Cross-Modal Attention-Mixer Based Structural-Functional Connectomics Fusion for Neuropsychiatric Disorder Diagnosis
Journal:
arXiv
Published Date:
May 21, 2025
Abstract
Gaining insights into the structural and functional mechanisms of the brain
has been a longstanding focus in neuroscience research, particularly in the
context of understanding and treating neuropsychiatric disorders such as
Schizophrenia (SZ). Nevertheless, most of the traditional multimodal deep
learning approaches fail to fully leverage the complementary characteristics of
structural and functional connectomics data to enhance diagnostic performance.
To address this issue, we proposed ConneX, a multimodal fusion method that
integrates cross-attention mechanism and multilayer perceptron (MLP)-Mixer for
refined feature fusion. Modality-specific backbone graph neural networks (GNNs)
were firstly employed to obtain feature representation for each modality. A
unified cross-modal attention network was then introduced to fuse these
embeddings by capturing intra- and inter-modal interactions, while MLP-Mixer
layers refined global and local features, leveraging higher-order dependencies
for end-to-end classification with a multi-head joint loss. Extensive
evaluations demonstrated improved performance on two distinct clinical
datasets, highlighting the robustness of our proposed framework.
