Optimizing Large Language Models for Detecting Symptoms of Comorbid Depression or Anxiety in Chronic Diseases: Insights from Patient Messages
Journal:
arXiv
Published Date:
Mar 14, 2025
Abstract
Patients with diabetes are at increased risk of comorbid depression or
anxiety, complicating their management. This study evaluated the performance of
large language models (LLMs) in detecting these symptoms from secure patient
messages. We applied multiple approaches, including engineered prompts,
systemic persona, temperature adjustments, and zero-shot and few-shot learning,
to identify the best-performing model and enhance performance. Three out of
five LLMs demonstrated excellent performance (over 90% of F-1 and accuracy),
with Llama 3.1 405B achieving 93% in both F-1 and accuracy using a zero-shot
approach. While LLMs showed promise in binary classification and handling
complex metrics like Patient Health Questionnaire-4, inconsistencies in
challenging cases warrant further real-life assessment. The findings highlight
the potential of LLMs to assist in timely screening and referrals, providing
valuable empirical knowledge for real-world triage systems that could improve
mental health care for patients with chronic diseases.
