Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation
Journal:
arXiv
Published Date:
May 21, 2025
Abstract
Pretrained generative models have opened new frontiers in brain decoding by
enabling the synthesis of realistic texts and images from non-invasive brain
recordings. However, the reliability of such outputs remains
questionable--whether they truly reflect semantic activation in the brain, or
are merely hallucinated by the powerful generative models. In this paper, we
focus on EEG-to-text decoding and address its hallucination issue through the
lens of posterior collapse. Acknowledging the underlying mismatch in
information capacity between EEG and text, we reframe the decoding task as
semantic summarization of core meanings rather than previously verbatim
reconstruction of stimulus texts. To this end, we propose the Generative
Language Inspection Model (GLIM), which emphasizes learning informative and
interpretable EEG representations to improve semantic grounding under
heterogeneous and small-scale data conditions. Experiments on the public ZuCo
dataset demonstrate that GLIM consistently generates fluent, EEG-grounded
sentences without teacher forcing. Moreover, it supports more robust evaluation
beyond text similarity, through EEG-text retrieval and zero-shot semantic
classification across sentiment categories, relation types, and corpus topics.
Together, our architecture and evaluation protocols lay the foundation for
reliable and scalable benchmarking in generative brain decoding.