Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation.

Journal: NeuroImage. Clinical
Published Date: Sep 6, 2019

Abstract

Machine learning-based imaging diagnostics has recently reached or even surpassed the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on 3D convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS), the most widespread autoimmune neuroinflammatory disease. MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients (n = 76) and healthy controls (n = 71). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of CNN models transparent, which could serve to justify classification decisions for clinical review, verify diagnosis-relevant features and potentially gather new disease knowledge.

Authors

  • Fabian Eitel
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Psychiatry and Psychotherapy, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Berlin Center for Advanced Neuroimaging, Bernstein Center for Computational Neuroscience, 10117 Berlin, Germany.
  • Emily Soehler
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Psychiatry and Psychotherapy, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Berlin Center for Advanced Neuroimaging, Bernstein Center for Computational Neuroscience, 10117 Berlin, Germany.
  • Judith Bellmann-Strobl
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universitt zu Berlin, Berlin Institute of Health (BIH), Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine, 10117 Berlin, Germany.
  • Alexander U Brandt
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany; Department of Neurology, University of California, Irvine, CA, USA.
  • Klemens Ruprecht
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany.
  • René M Giess
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany.
  • Joseph Kuchling
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universitt zu Berlin, Berlin Institute of Health (BIH), Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine, 10117 Berlin, Germany.
  • Susanna Asseyer
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universitt zu Berlin, Berlin Institute of Health (BIH), Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine, 10117 Berlin, Germany.
  • Martin Weygandt
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany.
  • John-Dylan Haynes
    Bernstein Center for Computational Neuroscience, Charité - Universitätsmedizin Berlin, 10117 Berlin, Germany; Berlin Center for Advanced Neuroimaging, Charité - Universitätsmedizin Berlin, 10117 Berlin, Germany; Humboldt-Universität zu Berlin, Berlin School of Mind and Brain and Institute of Psychology, 10099 Berlin, Germany; Technische Universität Dresden, SFB 940 Volition and Cognitive Control, 01069 Dresden, Germany.
  • Michael Scheel
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universitt zu Berlin, Berlin Institute of Health (BIH), Department of Neuroradiology, 10117 Berlin, Germany.
  • Friedemann Paul
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Neurology, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), NeuroCure Clinical Research Center, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universitt zu Berlin, Berlin Institute of Health (BIH), Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine, 10117 Berlin, Germany; Einstein Center for Digital Future Berlin, Germany.
  • Kerstin Ritter
    Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Department of Psychiatry and Psychotherapy, 10117 Berlin, Germany; Charité - Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health (BIH), Berlin Center for Advanced Neuroimaging, Bernstein Center for Computational Neuroscience, 10117 Berlin, Germany. Electronic address: kerstin.ritter@charite.de.

Keywords

External Resources

Popular Topics
Recent Journals