Enabling Rapid Shared Human-AI Mental Model Alignment via the After-Action Review
Journal:
arXiv
Published Date:
Mar 25, 2025
Abstract
In this work, we present two novel contributions toward improving research in
human-machine teaming (HMT): 1) a Minecraft testbed to accelerate testing and
deployment of collaborative AI agents and 2) a tool to allow users to revisit
and analyze behaviors within an HMT episode to facilitate shared mental model
development. Our browser-based Minecraft testbed allows for rapid testing of
collaborative agents in a continuous-space, real-time, partially-observable
environment with real humans without cumbersome setup typical to human-AI
interaction user studies. As Minecraft has an extensive player base and a rich
ecosystem of pre-built AI agents, we hope this contribution can help to
facilitate research quickly in the design of new collaborative agents and in
understanding different human factors within HMT. Our mental model alignment
tool facilitates user-led post-mission analysis by including video displays of
first-person perspectives of the team members (i.e., the human and AI) that can
be replayed, and a chat interface that leverages GPT-4 to provide answers to
various queries regarding the AI's experiences and model details.
