Multimodal deep learning for predicting in-hospital mortality in heart failure patients using longitudinal chest X-rays and electronic health records.

Journal: The international journal of cardiovascular imaging
PMID: 39786626

Abstract

Amid an aging global population, heart failure has become a leading cause of hospitalization among older people. Its high prevalence and mortality rates underscore the importance of accurate mortality prediction for swift disease progression assessment and better patient outcomes. The evolution of artificial intelligence (AI) presents new avenues for predicting heart failure mortality. Yet current research has predominantly leveraged structured data and unstructured clinical notes from electronic health records (EHR), underutilizing the prognostic value of chest X-rays (CXRs). This study aims to harness deep learning methodologies to explore the feasibility of enhancing the precision of predicting in-hospital all-cause mortality in heart failure patients using CXRs data. We propose a novel multimodal deep learning network based on the spatially and temporally decoupled Transformer (MN-STDT) for in-hospital mortality prediction in heart failure by integrating longitudinal CXRs and structured EHR data. The MN-STDT captures spatial and temporal information from CXRs through a Hybrid Spatial Encoder and a Distance-Aware Temporal Encoder, ultimately fusing features from both modalities for predictive modeling. Initial pre-training of the spatial encoder was conducted on CheXpert, followed by full model training and evaluation on the MIMIC-IV and MIMIC-CXR datasets for mortality prediction tasks. In a comprehensive view, the MN-STDT demonstrated the best performance, with an AUC-ROC of 0.8620, surpassing all baseline models. Comparative analysis revealed that the AUC-ROC of the multimodal model (0.8620) was significantly higher than that of models using only structured data (0.8166) or chest X-ray data alone (0.7479). This study demonstrates the value of CXRs in the prognosis of heart failure, showing that the combination of longitudinal CXRs with structured EHR data can significantly improve the accuracy of mortality prediction in heart failure. Feature importance analysis based on SHAP provides interpretable decision support, paving the way for potential clinical applications.

Authors

  • Dengao Li
    Technology Research Center of Spatial Information Network Engineering of Shanxi, Taiyuan, Shanxi, China.
  • Wen Xing
    College of Computer Science and Technology (College of Data Science), Taiyuan University of Technology, 30 Yingze West Street, Taiyuan, 030024, Shanxi, China.
  • Jumin Zhao
    College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China.
  • Changcheng Shi
    Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Science, Ningbo, Zhejiang 315201, P.R.China.
  • Fei Wang
    Department of Population Health Sciences, Weill Cornell Medicine, Cornell University, New York, NY, United States.

Keywords

External Resources

Popular Topics
Recent Journals