Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities.

Journal: Journal of biomedical informatics
Published Date: Aug 5, 2023

Abstract

OBJECTIVE: With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities.

Authors

  • Jinghui Liu
    School of Information, University of Michigan, Ann Arbor, Michigan, USA.
  • Daniel Capurro
    School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Centre for Digital Transformation of Health, Melbourne Medical School, The University of Melbourne, Victoria, Australia. Electronic address: dcapurro@unimelb.edu.au.
  • Anthony Nguyen
    Australian e-Health Research Centre, CSIRO, Brisbane, QLD, Australia.
  • Karin Verspoor
    Dept of Computing and Information Systems, School of Engineering, University of Melbourne, Melbourne, Australia.

Keywords

External Resources

Popular Topics
Recent Journals