Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis
Journal:
arXiv
Published Date:
Jul 9, 2025
Abstract
We propose a novel spatial-temporal graph Mamba (STG-Mamba) for the
music-guided dance video synthesis task, i.e., to translate the input music to
a dance video. STG-Mamba consists of two translation mappings:
music-to-skeleton translation and skeleton-to-video translation. In the
music-to-skeleton translation, we introduce a novel spatial-temporal graph
Mamba (STGM) block to effectively construct skeleton sequences from the input
music, capturing dependencies between joints in both the spatial and temporal
dimensions. For the skeleton-to-video translation, we propose a novel
self-supervised regularization network to translate the generated skeletons,
along with a conditional image, into a dance video. Lastly, we collect a new
skeleton-to-video translation dataset from the Internet, containing 54,944
video clips. Extensive experiments demonstrate that STG-Mamba achieves
significantly better results than existing methods.
