Dynamic Vision from EEG Brain Recordings: How much does EEG know?
Journal:
arXiv
Published Date:
May 27, 2025
Abstract
Reconstructing and understanding dynamic visual information (video) from
brain EEG recordings is challenging due to the non-stationary nature of EEG
signals, their low signal-to-noise ratio (SNR), and the limited availability of
EEG-Video stimulus datasets. Most recent studies have focused on reconstructing
static images from EEG recordings. In this work, we propose a framework to
reconstruct dynamic visual stimuli from EEG data and conduct an in-depth study
of the information encoded in EEG signals. Our approach first trains a feature
extraction network using a triplet-based contrastive learning strategy within
an EEG-video generation framework. The extracted EEG features are then used for
video synthesis with a modified StyleGAN-ADA, which incorporates temporal
information as conditioning. Additionally, we analyze how different brain
regions contribute to processing dynamic visual stimuli. Through several
empirical studies, we evaluate the effectiveness of our framework and
investigate how much dynamic visual information can be inferred from EEG
signals. The inferences we derive through our extensive studies would be of
immense value to future research on extracting visual dynamics from EEG.
