Accurate prediction of disease-risk factors from volumetric medical scans by a deep vision model pre-trained with 2D scans.

Journal: Nature biomedical engineering
Published Date: Oct 1, 2024

Abstract

The application of machine learning to tasks involving volumetric biomedical imaging is constrained by the limited availability of annotated datasets of three-dimensional (3D) scans for model training. Here we report a deep-learning model pre-trained on 2D scans (for which annotated data are relatively abundant) that accurately predicts disease-risk factors from 3D medical-scan modalities. The model, which we named SLIViT (for 'slice integration by vision transformer'), preprocesses a given volumetric scan into 2D images, extracts their feature map and integrates it into a single prediction. We evaluated the model in eight different learning tasks, including classification and regression for six datasets involving four volumetric imaging modalities (computed tomography, magnetic resonance imaging, optical coherence tomography and ultrasound). SLIViT consistently outperformed domain-specific state-of-the-art models and was typically as accurate as clinical specialists who had spent considerable time manually annotating the analysed scans. Automating diagnosis tasks involving volumetric scans may save valuable clinician hours, reduce data acquisition costs and duration, and help expedite medical research and clinical applications.

Authors

  • Oren Avram
    The Shmunis School of Biomedicine and Cancer Research, George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, Israel.
  • Berkin Durmus
    Department of Computer Science, University of California, Los Angeles, Los Angeles, CA, USA.
  • Nadav Rakocz
    Department of Computer Science, University of California, Los Angeles, CA, USA.
  • Giulia Corradetti
    Doheny Eye Institute, Pasadena, CA, USA.
  • Ulzee An
    Department of Computer Science, UCLA, Los Angeles, California, United States of America.
  • Muneeswar G Nittala
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Prerit Terway
    Department of Computational Medicine, University of California, Los Angeles, Los Angeles, CA, USA.
  • Akos Rudas
    Department of Computational Medicine, UCLA, Los Angeles, California, United States of America.
  • Zeyuan Johnson Chen
    Department of Computer Science, University of California, Los Angeles, Los Angeles, CA, USA.
  • Yu Wakatsuki
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Kazutaka Hirabayashi
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Swetha Velaga
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Liran Tiosano
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Federico Corvi
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Aditya Verma
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Ayesha Karamat
    Doheny Eye Institute, Pasadena, CA, USA.
  • Sophiana Lindenberg
    Doheny Eye Institute, Los Angeles, CA, 90033, USA.
  • Deniz Oncel
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Louay Almidani
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Victoria Hull
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Sohaib Fasih-Ahmad
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Houri Esmaeilkhanian
    Doheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USA.
  • Maxime Cannesson
    Department of Anesthesiology and Perioperative Medicine, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California.
  • Charles C Wykoff
    Retina Consultants of Texas, Retina Consultants of America, Houston, TX, USA.
  • Elior Rahmani
    Department of Computer Science, UCLA, Los Angeles, California, United States of America.
  • Corey W Arnold
    Department of Bioengineering; University of California, Los Angeles, CA.
  • Bolei Zhou
    Department of Computer Science, University of California, Los Angeles, CA, USA.
  • Noah Zaitlen
    Neurology Department, UCLA, Los Angeles, CA, USA.
  • Ilan Gronau
    School of Computer Science, Reichman University, Herzliya, Israel.
  • Sriram Sankararaman
    Computational Medicine, University of California, Los Angeles, Los Angeles, CA, USA.
  • Jeffrey N Chiang
    Department of Computational Medicine, UCLA, Los Angeles, California, United States of America.
  • SriniVas R Sadda
    Doheny Image Analysis Laboratory, Doheny Eye Institute, Los Angeles, CA, USA.
  • Eran Halperin
    Departments of Computer Science and Biomathmatics, UCLA Henry Samueli School of Engineering and Applied Science.

Keywords

External Resources

Popular Topics
Recent Journals