Designing VR Simulation System for Clinical Communication Training with LLMs-Based Embodied Conversational Agents
Journal:
arXiv
Published Date:
Mar 3, 2025
Abstract
VR simulation in Health Professions (HP) education demonstrates huge
potential, but fixed learning content with little customization limits its
application beyond lab environments. To address these limitations in the
context of VR for patient communication training, we conducted a user-centered
study involving semi-structured interviews with advanced HP students to
understand their challenges in clinical communication training and perceptions
of VR-based solutions. From this, we derived design insights emphasizing the
importance of realistic scenarios, simple interactions, and unpredictable
dialogues. Building on these insights, we developed the Virtual AI Patient
Simulator (VAPS), a novel VR system powered by Large Language Models (LLMs) and
Embodied Conversational Agents (ECAs), supporting dynamic and customizable
patient interactions for immersive learning. We also provided an example of how
clinical professors could use user-friendly design forms to create personalized
scenarios that align with course objectives in VAPS and discuss future
implications of integrating AI-driven technologies into VR education.
