Machine Learning for Workflow Applications in Screening Mammography: Systematic Review and Meta-Analysis.

Journal: Radiology
Published Date: Oct 19, 2021

Abstract

Background Advances in computer processing and improvements in data availability have led to the development of machine learning (ML) techniques for mammographic imaging. Purpose To evaluate the reported performance of stand-alone ML applications for screening mammography workflow. Materials and Methods Ovid Embase, Ovid Medline, Cochrane Central Register of Controlled Trials, Scopus, and Web of Science literature databases were searched for relevant studies published from January 2012 to September 2020. The study was registered with the PROSPERO International Prospective Register of Systematic Reviews (protocol no. CRD42019156016). Stand-alone technology was defined as a ML algorithm that can be used independently of a human reader. Studies were quality assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 and the Prediction Model Risk of Bias Assessment Tool, and reporting was evaluated using the Checklist for Artificial Intelligence in Medical Imaging. A primary meta-analysis included the top-performing algorithm and corresponding reader performance from which pooled summary estimates for the area under the receiver operating characteristic curve (AUC) were calculated using a bivariate model. Results Fourteen articles were included, which detailed 15 studies for stand-alone detection ( = 8) and triage ( = 7). Triage studies reported that 17%-91% of normal mammograms identified could be read by adapted screening, while "missing" an estimated 0%-7% of cancers. In total, an estimated 185 252 cases from three countries with more than 39 readers were included in the primary meta-analysis. The pooled sensitivity, specificity, and AUC was 75.4% (95% CI: 65.6, 83.2; = .11), 90.6% (95% CI: 82.9, 95.0; = .40), and 0.89 (95% CI: 0.84, 0.98), respectively, for algorithms, and 73.0% (95% CI: 60.7, 82.6), 88.6% (95% CI: 72.4, 95.8), and 0.85 (95% CI: 0.78, 0.97), respectively, for readers. Conclusion Machine learning (ML) algorithms that demonstrate a stand-alone application in mammographic screening workflows achieve or even exceed human reader detection performance and improve efficiency. However, this evidence is from a small number of retrospective studies. Therefore, further rigorous independent external prospective testing of ML algorithms to assess performance at preassigned thresholds is required to support these claims. ©RSNA, 2021 See also the editorial by Whitman and Moseley in this issue.

Authors

  • Sarah E Hickman
    From the Department of Radiology (S.E.H., R.W., G.C.B., J.W.M., F.J.G.) and Department of Medicine (E.P.V.L., Y.R.I., C.M.L.), University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, England; Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals National Health Service Foundation Trust, Cambridge, England (R.W., F.J.G.); Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria (R.W.); Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, England (A.I.A.R.); and Norwich Medical School, University of East Anglia, Norwich, England (J.W.M.).
  • Ramona Woitek
    Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK; Cancer Research UK Cambridge Centre, University of Cambridge, Cambridge CB2 0RE, UK; Department of Biomedical Imaging and Image-guided Therapy, Medical University Vienna, Vienna 1090, Austria. Electronic address: rw585@cam.ac.uk.
  • Elizabeth Phuong Vi Le
    From the Department of Radiology (S.E.H., R.W., G.C.B., J.W.M., F.J.G.) and Department of Medicine (E.P.V.L., Y.R.I., C.M.L.), University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, England; Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals National Health Service Foundation Trust, Cambridge, England (R.W., F.J.G.); Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria (R.W.); Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, England (A.I.A.R.); and Norwich Medical School, University of East Anglia, Norwich, England (J.W.M.).
  • Yu Ri Im
    From the Department of Radiology (S.E.H., R.W., G.C.B., J.W.M., F.J.G.) and Department of Medicine (E.P.V.L., Y.R.I., C.M.L.), University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, England; Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals National Health Service Foundation Trust, Cambridge, England (R.W., F.J.G.); Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria (R.W.); Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, England (A.I.A.R.); and Norwich Medical School, University of East Anglia, Norwich, England (J.W.M.).
  • Carina Mouritsen Luxhøj
    From the Department of Radiology (S.E.H., R.W., G.C.B., J.W.M., F.J.G.) and Department of Medicine (E.P.V.L., Y.R.I., C.M.L.), University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, England; Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals National Health Service Foundation Trust, Cambridge, England (R.W., F.J.G.); Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria (R.W.); Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, England (A.I.A.R.); and Norwich Medical School, University of East Anglia, Norwich, England (J.W.M.).
  • Angelica I Aviles-Rivero
    Department of Pure Mathematics & Mathematical Statistics, University of Cambridge, Cambridge, UK. ai323@cam.ac.uk.
  • Gabrielle C Baxter
    From the Department of Radiology (S.E.H., R.W., G.C.B., J.W.M., F.J.G.) and Department of Medicine (E.P.V.L., Y.R.I., C.M.L.), University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, England; Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals National Health Service Foundation Trust, Cambridge, England (R.W., F.J.G.); Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria (R.W.); Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, England (A.I.A.R.); and Norwich Medical School, University of East Anglia, Norwich, England (J.W.M.).
  • James W MacKay
    Department of Radiology, University of Cambridge, Cambridge, United Kingdom; Norwich Medical School, University of East Anglia, Norwich, United Kingdom.
  • Fiona J Gilbert
    Department of Radiology, University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom; NIHR Cambridge Biomedical Research Center, Cambridge, United Kingdom.

Keywords

External Resources

Popular Topics
Recent Journals