MixerMDM: Learnable Composition of Human Motion Diffusion Models
Journal:
arXiv
Published Date:
Apr 1, 2025
Abstract
Generating human motion guided by conditions such as textual descriptions is
challenging due to the need for datasets with pairs of high-quality motion and
their corresponding conditions. The difficulty increases when aiming for finer
control in the generation. To that end, prior works have proposed to combine
several motion diffusion models pre-trained on datasets with different types of
conditions, thus allowing control with multiple conditions. However, the
proposed merging strategies overlook that the optimal way to combine the
generation processes might depend on the particularities of each pre-trained
generative model and also the specific textual descriptions. In this context,
we introduce MixerMDM, the first learnable model composition technique for
combining pre-trained text-conditioned human motion diffusion models. Unlike
previous approaches, MixerMDM provides a dynamic mixing strategy that is
trained in an adversarial fashion to learn to combine the denoising process of
each model depending on the set of conditions driving the generation. By using
MixerMDM to combine single- and multi-person motion diffusion models, we
achieve fine-grained control on the dynamics of every person individually, and
also on the overall interaction. Furthermore, we propose a new evaluation
technique that, for the first time in this task, measures the interaction and
individual quality by computing the alignment between the mixed generated
motions and their conditions as well as the capabilities of MixerMDM to adapt
the mixing throughout the denoising process depending on the motions to mix.
