3D-Telepathy: Reconstructing 3D Objects from EEG Signals
Journal:
arXiv
Published Date:
Jun 27, 2025
Abstract
Reconstructing 3D visual stimuli from Electroencephalography (EEG) data holds
significant potential for applications in Brain-Computer Interfaces (BCIs) and
aiding individuals with communication disorders. Traditionally, efforts have
focused on converting brain activity into 2D images, neglecting the translation
of EEG data into 3D objects. This limitation is noteworthy, as the human brain
inherently processes three-dimensional spatial information regardless of
whether observing 2D images or the real world. The neural activities captured
by EEG contain rich spatial information that is inevitably lost when
reconstructing only 2D images, thus limiting its practical applications in BCI.
The transition from EEG data to 3D object reconstruction faces considerable
obstacles. These include the presence of extensive noise within EEG signals and
a scarcity of datasets that include both EEG and 3D information, which
complicates the extraction process of 3D visual data. Addressing this
challenging task, we propose an innovative EEG encoder architecture that
integrates a dual self-attention mechanism. We use a hybrid training strategy
to train the EEG Encoder, which includes cross-attention, contrastive learning,
and self-supervised learning techniques. Additionally, by employing stable
diffusion as a prior distribution and utilizing Variational Score Distillation
to train a neural radiation field, we successfully generate 3D objects with
similar content and structure from EEG data.
