DeVisE: Behavioral Testing of Medical Large Language Models
Journal:
arXiv
Published Date:
Jun 18, 2025
Abstract
Large language models (LLMs) are increasingly used in clinical decision
support, yet current evaluation methods often fail to distinguish genuine
medical reasoning from superficial patterns. We introduce DeVisE (Demographics
and Vital signs Evaluation), a behavioral testing framework for probing
fine-grained clinical understanding. We construct a dataset of ICU discharge
notes from MIMIC-IV, generating both raw (real-world) and template-based
(synthetic) versions with controlled single-variable counterfactuals targeting
demographic (age, gender, ethnicity) and vital sign attributes. We evaluate
five LLMs spanning general-purpose and medically fine-tuned variants, under
both zero-shot and fine-tuned settings. We assess model behavior via (1)
input-level sensitivity - how counterfactuals alter the likelihood of a note;
and (2) downstream reasoning - how they affect predicted hospital
length-of-stay. Our results show that zero-shot models exhibit more coherent
counterfactual reasoning patterns, while fine-tuned models tend to be more
stable yet less responsive to clinically meaningful changes. Notably,
demographic factors subtly but consistently influence outputs, emphasizing the
importance of fairness-aware evaluation. This work highlights the utility of
behavioral testing in exposing the reasoning strategies of clinical LLMs and
informing the design of safer, more transparent medical AI systems.
