Effect of data leakage in brain MRI classification using 2D convolutional neural networks.

Journal: Scientific reports
Published Date: Nov 19, 2021

Abstract

In recent years, 2D convolutional neural networks (CNNs) have been extensively used to diagnose neurological diseases from magnetic resonance imaging (MRI) data due to their potential to discern subtle and intricate patterns. Despite the high performances reported in numerous studies, developing CNN models with good generalization abilities is still a challenging task due to possible data leakage introduced during cross-validation (CV). In this study, we quantitatively assessed the effect of a data leakage caused by 3D MRI data splitting based on a 2D slice-level using three 2D CNN models to classify patients with Alzheimer's disease (AD) and Parkinson's disease (PD). Our experiments showed that slice-level CV erroneously boosted the average slice level accuracy on the test set by 30% on Open Access Series of Imaging Studies (OASIS), 29% on Alzheimer's Disease Neuroimaging Initiative (ADNI), 48% on Parkinson's Progression Markers Initiative (PPMI) and 55% on a local de-novo PD Versilia dataset. Further tests on a randomly labeled OASIS-derived dataset produced about 96% of (erroneous) accuracy (slice-level split) and 50% accuracy (subject-level split), as expected from a randomized experiment. Overall, the extent of the effect of an erroneous slice-based CV is severe, especially for small datasets.

Authors

  • Ekin Yagis
    School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK.
  • Selamawet Workalemahu Atnafu
    Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", University of Bologna, Via dell'Università 50, 47521, Cesena, Italy.
  • Alba García Seco de Herrera
    School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK.
  • Chiara Marzi
    Department of Electrical, Electronic, and Information Engineering 'Guglielmo Marconi', University of Bologna, Cesena, Italy.
  • Riccardo Scheda
    Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", University of Bologna, Via dell'Università 50, 47521, Cesena, Italy.
  • Marco Giannelli
    Unit of Medical Physics, Pisa University Hospital "Azienda Ospedaliero-Universitaria Pisana", via Roma 67, 56126 Pisa, Italy.
  • Carlo Tessa
    U.O.C. Radiologia Apuane e Lunigiana, Ospedale Apuane, Massa.
  • Luca Citi
    School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom.
  • Stefano Diciotti
    Department of Electrical, Electronic, and Information Engineering 'Guglielmo Marconi', University of Bologna, Cesena, Italy.

Keywords

External Resources

Popular Topics
Recent Journals