Accuracy of an Artificial Intelligence System for Interval Breast Cancer Detection at Screening Mammography.

Journal: Radiology
Published Date: Aug 1, 2024

Abstract

Background Artificial intelligence (AI) systems can be used to identify interval breast cancers, although the localizations are not always accurate. Purpose To evaluate AI localizations of interval cancers (ICs) on screening mammograms by IC category and histopathologic characteristics. Materials and Methods A screening mammography data set (median patient age, 57 years [IQR, 52-64 years]) that had been assessed by two human readers from January 2011 to December 2018 was retrospectively analyzed using a commercial AI system. The AI outputs were lesion locations (heatmaps) and the highest per-lesion risk score (range, 0-100) assigned to each case. AI heatmaps were considered false positive (FP) if they occurred on normal screening mammograms or on IC screening mammograms (ie, in patients subsequently diagnosed with IC) but outside the cancer boundary. A panel of consultant radiology experts classified ICs as normal or benign (true negative [TN]), uncertain (minimal signs of malignancy [MS]), or suspicious (false negative [FN]). Several specificity and sensitivity thresholds were applied. Mann-Whitney tests, Kruskal-Wallis tests, and χ tests were used to compare groups. Results A total of 2052 screening mammograms (514 ICs and 1548 normal mammograms) were included. The median AI risk score was 50 (IQR, 32-82) for TN ICs, 76 (IQR, 41-90) for ICs with MS, and 89 (IQR, 81-95) for FN ICs ( = .005). Higher median AI scores were observed for invasive tumors (62 [IQR, 39-88]) than for noninvasive tumors (33 [IQR, 20-55]; < .01) and for high-grade (grade 2-3) tumors (62 [IQR, 40-87]) than for low-grade (grade 0-1) tumors (45 [IQR, 26-81]; = .02). At the 96% specificity threshold, the AI algorithm flagged 121 of 514 (23.5%) ICs and correctly localized the IC in 93 of 121 (76.9%) cases, with 48 FP heatmaps on the mammograms for ICs (rate, 0.093 per case) and 74 FP heatmaps on normal mammograms (rate, 0.048 per case). The AI algorithm correctly localized a lower proportion of TN ICs (54 of 427; 12.6%) than ICs with MS (35 of 76; 46%) and FN ICs (four of eight; 50% [95% CI: 13, 88]; < .001). The AI algorithm localized a higher proportion of node-positive than node-negative cancers ( = .03). However, no evidence of a difference by cancer type ( = .09), grade ( = .27), or hormone receptor status ( = .12) was found. At 89.8% specificity and 79% sensitivity thresholds, AI detection increased to 181 (35.2%) and 256 (49.8%) of the 514 ICs, respectively, with FP heatmaps on 158 (10.2%) and 307 (19.8%) of the 1548 normal mammograms. Conclusion Use of a standalone AI system improved early cancer detection by correctly identifying some cancers missed by two human readers, with no differences based on histopathologic features except for node-positive cancers. © RSNA, 2024 .

Authors

  • Muzna Nanaa
    From the Department of Radiology, University of Cambridge School of Clinical Medicine, Box 218, Level 5, Cambridge Biomedical Campus, Cambridge CB2 0QQ, UK (S.E.H., N.R.P., Y.H., A.N.P., M.N., F.J.G.); University of Cambridge School of Clinical Medicine, Cambridge, UK (M.I.A, A.S.); Department of Radiology, Barts Health NHS Trust, The Royal London Hospital, London, UK (S.E.H.); Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK (R.T.B., A.N.P., F.J.G.); EPSRC Cambridge Mathematics of Information in Healthcare Hub, University of Cambridge, Cambridge, UK (Y.H.); Peel & Schriek Consulting, London, UK (S.H.); Department of Radiology, Norfolk and Norwich University Hospital, Norwich, UK (B.K., A.J.); and University of East Anglia, Norwich Research Park, Norwich, UK (B.K.).
  • Vaishnavi O Gupta
    From the Department of Radiology, School of Clinical Medicine, University of Cambridge, Box 218, Level 5, Cambridge Biomedical Campus, Cambridge CB2 0QQ, England (M.N., V.O.G., S.E.H., I.A., N.R.P., O.A., Y.H., A.N.P., F.J.G.); Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, England (M.N., I.A., R.B., A.N.P., F.J.G.); and Department of Radiology, The Royal London Hospital, Barts Health NHS Trust, London, England (S.E.H.).
  • Sarah E Hickman
    From the Department of Radiology (S.E.H., R.W., G.C.B., J.W.M., F.J.G.) and Department of Medicine (E.P.V.L., Y.R.I., C.M.L.), University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, England; Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals National Health Service Foundation Trust, Cambridge, England (R.W., F.J.G.); Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria (R.W.); Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, England (A.I.A.R.); and Norwich Medical School, University of East Anglia, Norwich, England (J.W.M.).
  • Iris Allajbeu
    Department of Radiology, University of Cambridge, Box 218, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, UK.
  • Nicholas R Payne
    From the Department of Radiology, University of Cambridge School of Clinical Medicine, Box 218, Level 5, Cambridge Biomedical Campus, Cambridge CB2 0QQ, UK (S.E.H., N.R.P., Y.H., A.N.P., M.N., F.J.G.); University of Cambridge School of Clinical Medicine, Cambridge, UK (M.I.A, A.S.); Department of Radiology, Barts Health NHS Trust, The Royal London Hospital, London, UK (S.E.H.); Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK (R.T.B., A.N.P., F.J.G.); EPSRC Cambridge Mathematics of Information in Healthcare Hub, University of Cambridge, Cambridge, UK (Y.H.); Peel & Schriek Consulting, London, UK (S.H.); Department of Radiology, Norfolk and Norwich University Hospital, Norwich, UK (B.K., A.J.); and University of East Anglia, Norwich Research Park, Norwich, UK (B.K.).
  • Otso Arponen
    Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland. otso.arponen@tuni.fi.
  • Richard Black
    From the Department of Radiology, School of Clinical Medicine, University of Cambridge, Box 218, Level 5, Cambridge Biomedical Campus, Cambridge CB2 0QQ, England (M.N., V.O.G., S.E.H., I.A., N.R.P., O.A., Y.H., A.N.P., F.J.G.); Department of Radiology, Addenbrooke's Hospital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, England (M.N., I.A., R.B., A.N.P., F.J.G.); and Department of Radiology, The Royal London Hospital, Barts Health NHS Trust, London, England (S.E.H.).
  • Yuan Huang
    School of Electrical Engineering and Information, Sichuan University, Chengdu 610065, China.
  • Andrew N Priest
    Department of Radiology, NIHR Cambridge Biomedical Resource Centre, Cambridge University Hospital, Cambridge, UK.
  • Fiona J Gilbert
    Department of Radiology, University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom; NIHR Cambridge Biomedical Research Center, Cambridge, United Kingdom.

Keywords

External Resources

Popular Topics
Recent Journals