Uncertainty-aware abstention in medical diagnosis based on medical texts
Journal:
arXiv
Published Date:
Feb 25, 2025
Abstract
This study addresses the critical issue of reliability for AI-assisted
medical diagnosis. We focus on the selection prediction approach that allows
the diagnosis system to abstain from providing the decision if it is not
confident in the diagnosis. Such selective prediction (or abstention)
approaches are usually based on the modeling predictive uncertainty of machine
learning models involved.
This study explores uncertainty quantification in machine learning models for
medical text analysis, addressing diverse tasks across multiple datasets. We
focus on binary mortality prediction from textual data in MIMIC-III,
multi-label medical code prediction using ICD-10 codes from MIMIC-IV, and
multi-class classification with a private outpatient visits dataset.
Additionally, we analyze mental health datasets targeting depression and
anxiety detection, utilizing various text-based sources, such as essays, social
media posts, and clinical descriptions.
In addition to comparing uncertainty methods, we introduce HUQ-2, a new
state-of-the-art method for enhancing reliability in selective prediction
tasks. Our results provide a detailed comparison of uncertainty quantification
methods. They demonstrate the effectiveness of HUQ-2 in capturing and
evaluating uncertainty, paving the way for more reliable and interpretable
applications in medical text analysis.
