Embodied Escaping: End-to-End Reinforcement Learning for Robot Navigation in Narrow Environment
Journal:
arXiv
Published Date:
Mar 5, 2025
Abstract
Autonomous navigation is a fundamental task for robot vacuum cleaners in
indoor environments. Since their core function is to clean entire areas, robots
inevitably encounter dead zones in cluttered and narrow scenarios. Existing
planning methods often fail to escape due to complex environmental constraints,
high-dimensional search spaces, and high difficulty maneuvers. To address these
challenges, this paper proposes an embodied escaping model that leverages
reinforcement learning-based policy with an efficient action mask for dead zone
escaping. To alleviate the issue of the sparse reward in training, we introduce
a hybrid training policy that improves learning efficiency. In handling
redundant and ineffective action options, we design a novel action
representation to reshape the discrete action space with a uniform turning
radius. Furthermore, we develop an action mask strategy to select valid action
quickly, balancing precision and efficiency. In real-world experiments, our
robot is equipped with a Lidar, IMU, and two-wheel encoders. Extensive
quantitative and qualitative experiments across varying difficulty levels
demonstrate that our robot can consistently escape from challenging dead zones.
Moreover, our approach significantly outperforms compared path planning and
reinforcement learning methods in terms of success rate and collision
avoidance.
