Simultaneous Estimation of Manipulation Skill and Hand Grasp Force from Forearm Ultrasound Images
Journal:
arXiv
Published Date:
Feb 1, 2025
Abstract
Accurate estimation of human hand configuration and the forces they exert is
critical for effective teleoperation and skill transfer in robotic
manipulation. A deeper understanding of human interactions with objects can
further enhance teleoperation performance. To address this need, researchers
have explored methods to capture and translate human manipulation skills and
applied forces to robotic systems. Among these, biosignal-based approaches,
particularly those using forearm ultrasound data, have shown significant
potential for estimating hand movements and finger forces. In this study, we
present a method for simultaneously estimating manipulation skills and applied
hand force using forearm ultrasound data. Data collected from seven
participants were used to train deep learning models for classifying
manipulation skills and estimating grasp force. Our models achieved an average
classification accuracy of 94.87 percent plus or minus 10.16 percent for
manipulation skills and an average root mean square error (RMSE) of 0.51 plus
or minus 0.19 Newtons for force estimation, as evaluated using five-fold
cross-validation. These results highlight the effectiveness of forearm
ultrasound in advancing human-machine interfacing and robotic teleoperation for
complex manipulation tasks. This work enables new and effective possibilities
for human-robot skill transfer and tele-manipulation, bridging the gap between
human dexterity and robotic control.
