Feel the Force: Contact-Driven Learning from Humans
Journal:
arXiv
Published Date:
Jun 2, 2025
Abstract
Controlling fine-grained forces during manipulation remains a core challenge
in robotics. While robot policies learned from robot-collected data or
simulation show promise, they struggle to generalize across the diverse range
of real-world interactions. Learning directly from humans offers a scalable
solution, enabling demonstrators to perform skills in their natural embodiment
and in everyday environments. However, visual demonstrations alone lack the
information needed to infer precise contact forces. We present FeelTheForce
(FTF): a robot learning system that models human tactile behavior to learn
force-sensitive manipulation. Using a tactile glove to measure contact forces
and a vision-based model to estimate hand pose, we train a closed-loop policy
that continuously predicts the forces needed for manipulation. This policy is
re-targeted to a Franka Panda robot with tactile gripper sensors using shared
visual and action representations. At execution, a PD controller modulates
gripper closure to track predicted forces-enabling precise, force-aware
control. Our approach grounds robust low-level force control in scalable human
supervision, achieving a 77% success rate across 5 force-sensitive manipulation
tasks. Code and videos are available at https://feel-the-force-ftf.github.io.
