Deep learning from multiple experts improves identification of amyloid neuropathologies.

Journal: Acta neuropathologica communications
PMID: 35484610

Abstract

Pathologists can label pathologies differently, making it challenging to yield consistent assessments in the absence of one ground truth. To address this problem, we present a deep learning (DL) approach that draws on a cohort of experts, weighs each contribution, and is robust to noisy labels. We collected 100,495 annotations on 20,099 candidate amyloid beta neuropathologies (cerebral amyloid angiopathy (CAA), and cored and diffuse plaques) from three institutions, independently annotated by five experts. DL methods trained on a consensus-of-two strategy yielded 12.6-26% improvements by area under the precision recall curve (AUPRC) when compared to those that learned individualized annotations. This strategy surpassed individual-expert models, even when unfairly assessed on benchmarks favoring them. Moreover, ensembling over individual models was robust to hidden random annotators. In blind prospective tests of 52,555 subsequent expert-annotated images, the models labeled pathologies like their human counterparts (consensus model AUPRC = 0.74 cored; 0.69 CAA). This study demonstrates a means to combine multiple ground truths into a common-ground DL model that yields consistent diagnoses informed by multiple and potentially variable expert opinions.

Authors

  • Daniel R Wong
    Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, 94158, USA.
  • Ziqi Tang
    Department of Pharmaceutical Chemistry, Department of Bioengineering and Therapeutic Sciences, Institute for Neurodegenerative Diseases, and Bakar Computational Health Sciences Institute, University of California, San Francisco, 675 Nelson Rising Ln Box 0518, San Francisco, CA, 94143, USA.
  • Nicholas C Mew
    Department of Pharmaceutical Chemistry, Department of Bioengineering and Therapeutic Sciences, Bakar Computational Health Sciences Institute, Kavli Institute for Fundamental Neuroscience, Institute for Neurodegenerative Diseases, University of California, San Francisco, 675 Nelson Rising Ln NS 416A, San Francisco, California 94143, United States.
  • Sakshi Das
    Department of Pathology and Laboratory Medicine, School of Medicine, University of California, Davis, Sacramento, CA, 95817, USA.
  • Justin Athey
    Department of Pathology and Laboratory Medicine, School of Medicine, University of California, Davis, Sacramento, CA, 95817, USA.
  • Kirsty E McAleese
    Translation and Clinical Research Institute, Newcastle University, Newcastle, UK.
  • Julia K Kofler
    Department of Pathology, University of Pittsburgh Medical Center, Pittsburgh, PA, 15260, USA.
  • Margaret E Flanagan
    Department of Pathology, Northwestern University, Evanston, IL, 60208, USA.
  • Ewa Borys
    Department of Pathology, Loyola University Medical Center, Maywood, IL, 60153, USA.
  • Charles L White
    Neuropathology Laboratory, Department of Pathology, UT Southwestern Medical Center, Dallas, TX, 75390, USA.
  • Atul J Butte
    Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, CA.
  • Brittany N Dugger
    Department of Pathology and Laboratory Medicine, University of California-Davis School of Medicine, 3400A Research Building III Sacramento, Davis, CA, 95817, USA. bndugger@ucdavis.edu.
  • Michael J Keiser
    Department of Pharmaceutical Chemistry, Department of Bioengineering and Therapeutic Sciences, Institute for Neurodegenerative Diseases and Bakar Institute for Computational Health Sciences , University of California-San Francisco , 675 Nelson Rising Lane , San Francisco , California 94158 , United States.

Keywords

External Resources

Popular Topics
Recent Journals