Deep Learning for Scalp-Level Nonlinear Source Separation in Electroencephalogram: A Comparative Evaluation of Spatiotemporal Architectures.

Electroencephalography (EEG) is a common technique to measure field potentials of various brain regions, and event-related potentials (ERPs) reflect stimulus-or response-locked brain responses that are spatially distributed across the scalp due to volume conduction. However, accurately capturing stimulus specific ERPs from EEG remains a major challenge due to complex mixtures of ERPs from multiple stimulus sources. This study addresses the challenge of separating nonlinearly mixed ERPs in EEG data using synthetic and experimental datasets. We simulated synthetic ERP datasets with known ground truth, by projecting source signals through a linear forward model, followed by various nonlinear mixing functions. We implemented different models, including tree-based deep forests, convolutional neural network (CNN), long short-term memory (LSTM), Transformer, and hybrid CNN-LSTM architectures to recover the individual source-specific scalp ERP projections. Performance was assessed via 2D Pearson correlation between the predicted ERPs and the ground truth as well as the source localization accuracy. Our results showed that model architectures integrating spatial and temporal modeling, particularly CNN-LSTM combined models, achieved superior signal reconstruction and anatomical localization accuracy. These findings highlight the potential of deep learning methods for nonlinear source-projection separation in EEG data, and provide a framework for future applications under realistic neural constraints.