HeMA: A hierarchically enriched machine learning approach for managing false alarms in real time: A sepsis prediction case study.
Journal:
Computers in biology and medicine
Published Date:
Feb 17, 2021
Abstract
Early detection of sepsis can be life-saving. Machine learning models have shown great promise in early sepsis prediction when applied to patient physiological data in real-time. However, these existing models often under-perform in terms of positive predictive value, an important metric in clinical settings. This is especially the case when the models are applied to data with less than 50% sepsis prevalence, reflective of the incidence rate of sepsis on the floor or in the ICU. In this study, we develop HeMA, a hierarchically enriched machine learning approach for managing false alarms in real time, and conduct a case study for early sepsis prediction. Specifically, we develop a two-stage framework, where a first stage machine learning model is paired with statistical tests, particularly Kolmogorov-Smirnov tests, in the second stage, to predict whether a patient would develop sepsis. Compared with machine learning models alone, the framework results in an increase in specificity and positive predictive value, without compromising F1 score. In particular, the framework shows improved performance when applied to data with 50% and 25% sepsis prevalence, collected from a large hospital system in the US, resulting in up to 18% and 7% increase in specificity and positive predictive value, respectively. Despite the significant improvements observed, and although F1 score is not negatively affected, because of the up to 6% decrease in sensitivity, further improvements and pilot studies may be necessary before deploying the framework in a clinical setting. Finally, external validation conducted using a publicly available dataset produces similar results, validating that the proposed framework is generalizable.