TROI: Cross-Subject Pretraining with Sparse Voxel Selection for Enhanced fMRI Visual Decoding
Journal:
arXiv
Published Date:
Feb 1, 2025
Abstract
fMRI (functional Magnetic Resonance Imaging) visual decoding involves
decoding the original image from brain signals elicited by visual stimuli. This
often relies on manually labeled ROIs (Regions of Interest) to select brain
voxels. However, these ROIs can contain redundant information and noise,
reducing decoding performance. Additionally, the lack of automated ROI labeling
methods hinders the practical application of fMRI visual decoding technology,
especially for new subjects. This work presents TROI (Trainable Region of
Interest), a novel two-stage, data-driven ROI labeling method for cross-subject
fMRI decoding tasks, particularly when subject samples are limited. TROI
leverages labeled ROIs in the dataset to pretrain an image decoding backbone on
a cross-subject dataset, enabling efficient optimization of the input layer for
new subjects without retraining the entire model from scratch. In the first
stage, we introduce a voxel selection method that combines sparse mask training
and low-pass filtering to quickly generate the voxel mask and determine input
layer dimensions. In the second stage, we apply a learning rate rewinding
strategy to fine-tune the input layer for downstream tasks. Experimental
results on the same small sample dataset as the baseline method for brain
visual retrieval and reconstruction tasks show that our voxel selection method
surpasses the state-of-the-art method MindEye2 with an annotated ROI mask.
