DETERIO-LLM: enhancing traditional deterioration risk scores with clinical context and advanced reasoning.

OBJECTIVES: Accurate and timely prediction of physiological deterioration in hospitalized patients is essential, but existing deep learning models, which rely solely on structured electronic health record (EHR) data, generate high false-positive rates by missing crucial contextual information in clinical notes. We present DETERIO-LLM, a hybrid prediction model that selectively integrates large language models (LLMs) with a deep learning model (DETERIO) to reclassify borderline-risk alerts using narrative clinical notes. MATERIALS AND METHODS: This retrospective study used a cohort of 1000 inpatients (4.6% deterioration prevalence). DETERIO-LLM selectively analyzed narrative notes for alerts in the uncertainty range (scores 2.5-3.5). Performance was evaluated using sensitivity, positive predictive value (PPV), and F1 score, benchmarked against comparators including the best-established ML model, eCART. RESULTS: DETERIO-LLM significantly improved predictive precision over all comparators. At its optimal threshold, the model achieved a PPV of 30.6% and an F1 score of 37.3%, outperforming eCART. This selective integration reduced false-positive alerts by 46.5% within the uncertainty range while maintaining comparable sensitivity. DISCUSSION: This study demonstrates a practical, interpretable method for integrating LLM-derived contextual knowledge into existing structured EHR prediction workflows. By focusing LLM analysis on high-uncertainty alerts, DETERIO-LLM substantially improves clinical utility through a marked reduction in false-positive cases, which is critical for mitigating alert fatigue. CONCLUSION: Selective LLM integration with structured EHR models offers a viable, interpretable path to improving early warning systems in clinical care by leveraging narrative context to reduce alert fatigue while enhancing predictive performance.