CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operation
Journal:
arXiv
Published Date:
Jun 15, 2025
Abstract
In clinical operations, teamwork can be the crucial factor that determines
the final outcome. Prior studies have shown that sufficient collaboration is
the key factor that determines the outcome of an operation. To understand how
the team practices teamwork during the operation, we collected CliniDial from
simulations of medical operations. CliniDial includes the audio data and its
transcriptions, the simulated physiology signals of the patient manikins, and
how the team operates from two camera angles. We annotate behavior codes
following an existing framework to understand the teamwork process for
CliniDial. We pinpoint three main characteristics of our dataset, including its
label imbalances, rich and natural interactions, and multiple modalities, and
conduct experiments to test existing LLMs' capabilities on handling data with
these characteristics. Experimental results show that CliniDial poses
significant challenges to the existing models, inviting future effort on
developing methods that can deal with real-world clinical data. We open-source
the codebase at https://github.com/MichiganNLP/CliniDial
