Artificial intelligence for multimodal data integration in oncology.

Journal: Cancer cell
Published Date: Oct 10, 2022

Abstract

In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, and genomics to electronic health records. Current artificial intelligence (AI) models operate mainly in the realm of a single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. AI models are also capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion and association discovery. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges in clinical adoption and discuss emerging solutions.

Authors

  • Jana Lipkova
    Department of Informatics, Technische Universität München, Munich, Germany.
  • Richard J Chen
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
  • Bowen Chen
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
  • Ming Y Lu
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
  • Matteo Barbieri
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
  • Daniel Shao
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA.
  • Anurag J Vaidya
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA.
  • Chengkuan Chen
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
  • Luoting Zhuang
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA.
  • Drew F K Williamson
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
  • Muhammad Shaban
    Tissue Image Analytics Lab, Department of Computer Science, University of Warwick, Coventry, United Kingdom.
  • Tiffany Y Chen
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
  • Faisal Mahmood
    Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA. faisalmahmood@bwh.harvard.edu.

Keywords

External Resources

Popular Topics
Recent Journals