Sociodemographic bias in large language model clinical trial screening.

Journal: Journal of the American Medical Informatics Association : JAMIA
Published Date:

Abstract

OBJECTIVE: To assess whether large language model (LLM)-based clinical trial screening judgments vary by patient sociodemographic characteristics. MATERIALS AND METHODS: We conducted a cross-sectional evaluation of Phase II-III US adult randomized controlled trial (RCT) protocols (2023-2024). Physician-validated clinical vignettes were evaluated in a control version and 33 sociodemographic identity variants differing only by labels. Nine LLMs assessed eligibility and related domains. Mixed-effects models estimated adjusted differences vs control. RESULTS: Across 58 protocols and 5.3 million evaluations, eligibility judgments were largely stable across identities. Race and ethnicity showed minimal effects after accounting for socioeconomic status. Homelessness produced the largest negative eligibility shift and pronounced effects in adherence, resources, and trust. DISCUSSION AND CONCLUSION: LLMs applied explicit eligibility criteria consistently, but disparities emerged in domains requiring inference about behavior or resources, underscoring the need for careful deployment to promote fair trial access.

Authors

Keywords

No keywords available for this article.