DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulation
Journal:
arXiv
Published Date:
Mar 21, 2025
Abstract
Nonprehensile manipulation is crucial for handling objects that are too thin,
large, or otherwise ungraspable in unstructured environments. While
conventional planning-based approaches struggle with complex contact modeling,
learning-based methods have recently emerged as a promising alternative.
However, existing learning-based approaches face two major limitations: they
heavily rely on multi-view cameras and precise pose tracking, and they fail to
generalize across varying physical conditions, such as changes in object mass
and table friction. To address these challenges, we propose the
Dynamics-Adaptive World Action Model (DyWA), a novel framework that enhances
action learning by jointly predicting future states while adapting to dynamics
variations based on historical trajectories. By unifying the modeling of
geometry, state, physics, and robot actions, DyWA enables more robust policy
learning under partial observability. Compared to baselines, our method
improves the success rate by 31.5% using only single-view point cloud
observations in the simulation. Furthermore, DyWA achieves an average success
rate of 68% in real-world experiments, demonstrating its ability to generalize
across diverse object geometries, adapt to varying table friction, and
robustness in challenging scenarios such as half-filled water bottles and
slippery surfaces.
