Human sensory-musculoskeletal modeling and control of whole-body movements
Journal:
arXiv
Published Date:
May 29, 2025
Abstract
Coordinated human movement depends on the integration of multisensory inputs,
sensorimotor transformation, and motor execution, as well as sensory feedback
resulting from body-environment interaction. Building dynamic models of the
sensory-musculoskeletal system is essential for understanding movement control
and investigating human behaviours. Here, we report a human
sensory-musculoskeletal model, termed SMS-Human, that integrates precise
anatomical representations of bones, joints, and muscle-tendon units with
multimodal sensory inputs involving visual, vestibular, proprioceptive, and
tactile components. A stage-wise hierarchical deep reinforcement learning
framework was developed to address the inherent challenges of high-dimensional
control in musculoskeletal systems with integrated multisensory information.
Using this framework, we demonstrated the simulation of three representative
movement tasks, including bipedal locomotion, vision-guided object
manipulation, and human-machine interaction during bicycling. Our results
showed a close resemblance between natural and simulated human motor
behaviours. The simulation also revealed musculoskeletal dynamics that could
not be directly measured. This work sheds deeper insights into the sensorimotor
dynamics of human movements, facilitates quantitative understanding of human
behaviours in interactive contexts, and informs the design of systems with
embodied intelligence.
