RAMBO: RL-augmented Model-based Optimal Control for Whole-body Loco-manipulation
Journal:
arXiv
Published Date:
Apr 9, 2025
Abstract
Loco-manipulation -- coordinated locomotion and physical interaction with
objects -- remains a major challenge for legged robots due to the need for both
accurate force interaction and robustness to unmodeled dynamics. While
model-based controllers provide interpretable dynamics-level planning and
optimization, they are limited by model inaccuracies and computational cost. In
contrast, learning-based methods offer robustness while struggling with precise
modulation of interaction forces. We introduce RAMBO -- RL-Augmented
Model-Based Optimal Control -- a hybrid framework that integrates model-based
reaction force optimization using a simplified dynamics model and a feedback
policy trained with reinforcement learning. The model-based module generates
feedforward torques by solving a quadratic program, while the policy provides
feedback residuals to enhance robustness in control execution. We validate our
framework on a quadruped robot across a diverse set of real-world
loco-manipulation tasks -- such as pushing a shopping cart, balancing a plate,
and holding soft objects -- in both quadrupedal and bipedal walking. Our
experiments demonstrate that RAMBO enables precise manipulation while achieving
robust and dynamic locomotion, surpassing the performance of policies trained
with end-to-end scheme. In addition, our method enables flexible trade-off
between end-effector tracking accuracy with compliance.
