MBE-ARI: A Multimodal Dataset Mapping Bi-directional Engagement in Animal-Robot Interaction
Journal:
arXiv
Published Date:
Apr 11, 2025
Abstract
Animal-robot interaction (ARI) remains an unexplored challenge in robotics,
as robots struggle to interpret the complex, multimodal communication cues of
animals, such as body language, movement, and vocalizations. Unlike human-robot
interaction, which benefits from established datasets and frameworks,
animal-robot interaction lacks the foundational resources needed to facilitate
meaningful bidirectional communication. To bridge this gap, we present the
MBE-ARI (Multimodal Bidirectional Engagement in Animal-Robot Interaction), a
novel multimodal dataset that captures detailed interactions between a legged
robot and cows. The dataset includes synchronized RGB-D streams from multiple
viewpoints, annotated with body pose and activity labels across interaction
phases, offering an unprecedented level of detail for ARI research.
Additionally, we introduce a full-body pose estimation model tailored for
quadruped animals, capable of tracking 39 keypoints with a mean average
precision (mAP) of 92.7%, outperforming existing benchmarks in animal pose
estimation. The MBE-ARI dataset and our pose estimation framework lay a robust
foundation for advancing research in animal-robot interaction, providing
essential tools for developing perception, reasoning, and interaction
frameworks needed for effective collaboration between robots and animals. The
dataset and resources are publicly available at
https://github.com/RISELabPurdue/MBE-ARI/, inviting further exploration and
development in this critical area.
