Performance of a Deep Learning Algorithm Compared with Radiologic Interpretation for Lung Cancer Detection on Chest Radiographs in a Health Screening Population.

Journal: Radiology
Published Date: Sep 22, 2020

Abstract

Background The performance of a deep learning algorithm for lung cancer detection on chest radiographs in a health screening population is unknown. Purpose To validate a commercially available deep learning algorithm for lung cancer detection on chest radiographs in a health screening population. Materials and Methods Out-of-sample testing of a deep learning algorithm was retrospectively performed using chest radiographs from individuals undergoing a comprehensive medical check-up between July 2008 and December 2008 (validation test). To evaluate the algorithm performance for visible lung cancer detection, the area under the receiver operating characteristic curve (AUC) and diagnostic measures, including sensitivity and false-positive rate (FPR), were calculated. The algorithm performance was compared with that of radiologists using the McNemar test and the Moskowitz method. Additionally, the deep learning algorithm was applied to a screening cohort undergoing chest radiography between January 2008 and December 2012, and its performances were calculated. Results In a validation test comprising 10 285 radiographs from 10 202 individuals (mean age, 54 years ± 11 [standard deviation]; 5857 men) with 10 radiographs of visible lung cancers, the algorithm's AUC was 0.99 (95% confidence interval: 0.97, 1), and it showed comparable sensitivity (90% [nine of 10 radiographs]) to that of the radiologists (60% [six of 10 radiographs]; = .25) with a higher FPR (3.1% [319 of 10 275 radiographs] vs 0.3% [26 of 10 275 radiographs]; < .001). In the screening cohort of 100 525 chest radiographs from 50 070 individuals (mean age, 53 years ± 11; 28 090 men) with 47 radiographs of visible lung cancers, the algorithm's AUC was 0.97 (95% confidence interval: 0.95, 0.99), and its sensitivity and FPR were 83% (39 of 47 radiographs) and 3% (2999 of 100 478 radiographs), respectively. Conclusion A deep learning algorithm detected lung cancers on chest radiographs with a performance comparable to that of radiologists, which will be helpful for radiologists in healthy populations with a low prevalence of lung cancer. © RSNA, 2020 See also the editorial by Armato in this issue.

Authors

  • Jong Hyuk Lee
    From the Department of Radiology and Institute of Radiation Medicine, Seoul National University Hospital and College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Republic of Korea (J.G.N., E.J.H., J.M.G., C.M.P.); Lunit Incorporated, Seoul, Republic of Korea (S.P.); Department of Radiology, Armed Forces Seoul Hospital, Seoul, Republic of Korea (J.H.L.); Department of Radiology, Seoul National University Boramae Medical Center, Seoul, Republic of Korea (K.N.J.); Department of Radiology, National Cancer Center, Goyang, Republic of Korea (K.Y.L.); Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, Calif (T.H.V., J.H.S.); and Department of Industrial & Information Systems Engineering, Seoul National University of Science and Technology, Seoul, Republic of Korea (S.H.).
  • Hye Young Sun
    From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.).
  • Sunggyun Park
    From the Department of Radiology and Institute of Radiation Medicine, Seoul National University Hospital and College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Republic of Korea (J.G.N., E.J.H., J.M.G., C.M.P.); Lunit Incorporated, Seoul, Republic of Korea (S.P.); Department of Radiology, Armed Forces Seoul Hospital, Seoul, Republic of Korea (J.H.L.); Department of Radiology, Seoul National University Boramae Medical Center, Seoul, Republic of Korea (K.N.J.); Department of Radiology, National Cancer Center, Goyang, Republic of Korea (K.Y.L.); Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, Calif (T.H.V., J.H.S.); and Department of Industrial & Information Systems Engineering, Seoul National University of Science and Technology, Seoul, Republic of Korea (S.H.).
  • Hyungjin Kim
    Department of Radiology, Seoul National College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Korea (H.C., S.H.Y., S.J.P., C.M.P., J.H.L., H. Kim, E.J.H., S.J.Y., J.G.N., C.H.L., J.M.G.); CHESS Center, The First Hospital of Lanzhou University, Lanzhou, China (Q.X., J.L.); Department of Radiology, Seoul National University Bundang Hospital, Gyeonggi-do, Korea (K.H.L.); Department of Internal Medicine, Incheon Medical Center, Incheon, Korea (J.Y.K.); Department of Radiology, Seoul Medical Center, Seoul, Korea (Y.K.L.); Department of Radiology, National Medical Center, Seoul, Korea (H. Ko); Department of Radiology, Myongji Hospital, Gyeonggi-do, Korea (K.H.K.); and Department of Radiology, Chonnam National University Hospital, Gwanju, Korea (Y.H.K.).
  • Eui Jin Hwang
    Department of Radiology, Seoul National College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Korea (H.C., S.H.Y., S.J.P., C.M.P., J.H.L., H. Kim, E.J.H., S.J.Y., J.G.N., C.H.L., J.M.G.); CHESS Center, The First Hospital of Lanzhou University, Lanzhou, China (Q.X., J.L.); Department of Radiology, Seoul National University Bundang Hospital, Gyeonggi-do, Korea (K.H.L.); Department of Internal Medicine, Incheon Medical Center, Incheon, Korea (J.Y.K.); Department of Radiology, Seoul Medical Center, Seoul, Korea (Y.K.L.); Department of Radiology, National Medical Center, Seoul, Korea (H. Ko); Department of Radiology, Myongji Hospital, Gyeonggi-do, Korea (K.H.K.); and Department of Radiology, Chonnam National University Hospital, Gwanju, Korea (Y.H.K.).
  • Jin Mo Goo
    Department of Radiology, Seoul National College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Korea (H.C., S.H.Y., S.J.P., C.M.P., J.H.L., H. Kim, E.J.H., S.J.Y., J.G.N., C.H.L., J.M.G.); CHESS Center, The First Hospital of Lanzhou University, Lanzhou, China (Q.X., J.L.); Department of Radiology, Seoul National University Bundang Hospital, Gyeonggi-do, Korea (K.H.L.); Department of Internal Medicine, Incheon Medical Center, Incheon, Korea (J.Y.K.); Department of Radiology, Seoul Medical Center, Seoul, Korea (Y.K.L.); Department of Radiology, National Medical Center, Seoul, Korea (H. Ko); Department of Radiology, Myongji Hospital, Gyeonggi-do, Korea (K.H.K.); and Department of Radiology, Chonnam National University Hospital, Gwanju, Korea (Y.H.K.).
  • Chang Min Park
    Department of Radiology, Seoul National College of Medicine, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Korea (H.C., S.H.Y., S.J.P., C.M.P., J.H.L., H. Kim, E.J.H., S.J.Y., J.G.N., C.H.L., J.M.G.); CHESS Center, The First Hospital of Lanzhou University, Lanzhou, China (Q.X., J.L.); Department of Radiology, Seoul National University Bundang Hospital, Gyeonggi-do, Korea (K.H.L.); Department of Internal Medicine, Incheon Medical Center, Incheon, Korea (J.Y.K.); Department of Radiology, Seoul Medical Center, Seoul, Korea (Y.K.L.); Department of Radiology, National Medical Center, Seoul, Korea (H. Ko); Department of Radiology, Myongji Hospital, Gyeonggi-do, Korea (K.H.K.); and Department of Radiology, Chonnam National University Hospital, Gwanju, Korea (Y.H.K.).

Keywords

External Resources

Popular Topics
Recent Journals