Making Your Dreams A Reality: Decoding the Dreams into a Coherent Video Story from fMRI Signals
Journal:
arXiv
Published Date:
Jan 16, 2025
Abstract
This paper studies the brave new idea for Multimedia community, and proposes
a novel framework to convert dreams into coherent video narratives using fMRI
data. Essentially, dreams have intrigued humanity for centuries, offering
glimpses into our subconscious minds. Recent advancements in brain imaging,
particularly functional magnetic resonance imaging (fMRI), have provided new
ways to explore the neural basis of dreaming. By combining subjective dream
experiences with objective neurophysiological data, we aim to understand the
visual aspects of dreams and create complete video narratives. Our process
involves three main steps: reconstructing visual perception, decoding dream
imagery, and integrating dream stories. Using innovative techniques in fMRI
analysis and language modeling, we seek to push the boundaries of dream
research and gain deeper insights into visual experiences during sleep. This
technical report introduces a novel approach to visually decoding dreams using
fMRI signals and weaving dream visuals into narratives using language models.
We gather a dataset of dreams along with descriptions to assess the
effectiveness of our framework.
