Bi-LAT: Bilateral Control-Based Imitation Learning via Natural Language and Action Chunking with Transformers
Journal:
arXiv
Published Date:
Apr 2, 2025
Abstract
We present Bi-LAT, a novel imitation learning framework that unifies
bilateral control with natural language processing to achieve precise force
modulation in robotic manipulation. Bi-LAT leverages joint position, velocity,
and torque data from leader-follower teleoperation while also integrating
visual and linguistic cues to dynamically adjust applied force. By encoding
human instructions such as "softly grasp the cup" or "strongly twist the
sponge" through a multimodal Transformer-based model, Bi-LAT learns to
distinguish nuanced force requirements in real-world tasks. We demonstrate
Bi-LAT's performance in (1) unimanual cup-stacking scenario where the robot
accurately modulates grasp force based on language commands, and (2) bimanual
sponge-twisting task that requires coordinated force control. Experimental
results show that Bi-LAT effectively reproduces the instructed force levels,
particularly when incorporating SigLIP among tested language encoders. Our
findings demonstrate the potential of integrating natural language cues into
imitation learning, paving the way for more intuitive and adaptive human-robot
interaction. For additional material, please visit:
https://mertcookimg.github.io/bi-lat/
