Prospective validation and real-time implementation of an automated machine learning postoperative mortality prediction model.

Journal: British journal of anaesthesia
Published Date: Jan 17, 2026

Abstract

BACKGROUND: Machine learning prediction models require prospective validation to ensure implementation fidelity and feasibility. Our primary objective was to prospectively validate a previously reported postoperative mortality prediction model in inpatients undergoing surgery. Our secondary objective was to evaluate feasibility of a pilot clinical decision support tool. METHODS: We prospectively validated and implemented a random forest machine learning model trained to predict in-hospital mortality using data from a single academic medical centre. A reduced 32-feature model was implemented into the electronic health record (EHR) using a real-time data mart at the same institution. To assess model performance, the area under the receiver operating characteristic curve (AUROC), area under the curve precision-recall (AUCPR), and other performance measures were calculated. To assess feasibility, implementation workflow metrics were evaluated and a survey was administered to anaesthesiologists trained to use the pilot clinical decision support tool. RESULTS: The AUROC for the prospectively implemented model was 0.874 (95% confidence interval [CI] 0.860-0.887), and the AUCPR was 0.111. By comparison, the AUROC for the 58-feature model was 0.925 (95% CI 0.900-0.947), and for ASA physical status the AUROC was 0.814 (95% CI 0.802-0.827) and the AUCPR was 0.103. The implementation demonstrated feasibility through real-time data updates, automated transfer of model outputs to the EHR, and provider survey entries. CONCLUSIONS: This prospective validation and EHR implementation of a previously published random forest machine learning model predicting postoperative in-hospital mortality demonstrated acceptable real-world performance of the implemented model and feasibility of integrating such a system into clinical practice.

Authors

  • Theodora Wingert
    University of California Los Angeles, David Geffen School of Medicine, Los Angeles, CA, USA; Department of Anesthesiology and Perioperative Medicine, Ronald Reagan UCLA Medical Center, 757 Westwood Plaza, Suite 3325, Los Angeles, CA 90095-7403, USA. Electronic address: [email protected].
  • Tiffany Williams
    Department of Anesthesiology & Perioperative Medicine, David Geffen School of Medicine, University of California Los Angeles, Ronald Reagan UCLA Medical Center, 757 Westwood Plaza, Los Angeles, CA, 90095, USA.
  • Briana Syed
    University of Texas Medical Branch, John Sealy School of Medicine, Galveston, TX, USA.
  • Brian Hill
    Age Bold, Inc., Los Angeles, CA, USA.
  • Tristan Grogan
    Department of Medicine Statistics Core, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA.
  • Andrew Young
    Centre of Australian National Biodiversity Research, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Canberra, ACT 2601, Australia.
  • Zarah Antongiorgi
    Department of Anesthesiology & Perioperative Medicine, University of California Los Angeles, Los Angeles, CA, USA.
  • Valiollah Salari
    Department of Anesthesiology & Perioperative Medicine, University of California Los Angeles, Los Angeles, CA, USA.
  • Alexandre Joosten
    Department of Anesthesiology & Intensive Care, Hôpital De Bicêtre, Assistance Publique Hôpitaux de Paris (AP-HP), Paris, France.
  • Ira Hofer
  • Eran Halperin
    Departments of Computer Science and Biomathmatics, UCLA Henry Samueli School of Engineering and Applied Science.
  • Maxime Cannesson
    Department of Anesthesiology and Perioperative Medicine, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California.
  • Eilon Gabel

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals