Vision-to-Music Generation: A Survey
Journal:
arXiv
Published Date:
Mar 27, 2025
Abstract
Vision-to-music Generation, including video-to-music and image-to-music
tasks, is a significant branch of multimodal artificial intelligence
demonstrating vast application prospects in fields such as film scoring, short
video creation, and dance music synthesis. However, compared to the rapid
development of modalities like text and images, research in vision-to-music is
still in its preliminary stage due to its complex internal structure and the
difficulty of modeling dynamic relationships with video. Existing surveys focus
on general music generation without comprehensive discussion on
vision-to-music. In this paper, we systematically review the research progress
in the field of vision-to-music generation. We first analyze the technical
characteristics and core challenges for three input types: general videos,
human movement videos, and images, as well as two output types of symbolic
music and audio music. We then summarize the existing methodologies on
vision-to-music generation from the architecture perspective. A detailed review
of common datasets and evaluation metrics is provided. Finally, we discuss
current challenges and promising directions for future research. We hope our
survey can inspire further innovation in vision-to-music generation and the
broader field of multimodal generation in academic research and industrial
applications. To follow latest works and foster further innovation in this
field, we are continuously maintaining a GitHub repository at
https://github.com/wzk1015/Awesome-Vision-to-Music-Generation.
