CodeBrain: Imputing Any Brain MRI via Modality- and Instance-Specific Codes
Journal:
arXiv
Published Date:
Jan 30, 2025
Abstract
Unified MRI imputation, which can adapt to diverse imputation scenarios, is
highly desirable as it reduces scanning costs and provides comprehensive MRI
information for improved clinical diagnosis. Existing unified MRI imputation
methods either rely on specific prompts to guide their transformation network
or require multiple modality-specific modules. However, these approaches
struggle to capture large modality and instance variations or become too
complex to generalize effectively. To address these limitations, we propose
CodeBrain, a fundamentally different pipeline for unified brain MRI imputation.
Our key idea is to reframe various inter-modality transformations as a
full-modality code prediction task via a two-stage framework. In the first
stage, CodeBrain reconstructs a target modality from any other modalities by
learning a compact scalar-quantized code for each instance and modality. Any
target modality can then be reconstructed with high fidelity by combining the
corresponding code with shared features extracted from any available modality.
In the second stage, a projection encoder is trained to predict full-modality
compact codes from any incomplete MRI samples, effectively simulating various
imputation scenarios. We evaluate our CodeBrain on two public brain MRI
datasets (i.e., IXI and BraTS 2023). Extensive experiments demonstrate that
CodeBrain outperforms state-of-the-art methods, setting a new benchmark for
unified brain MRI imputation. Our code will be released.
