Assistive AI in Lung Cancer Screening: A Retrospective Multinational Study in the United States and Japan.

Journal: Radiology. Artificial intelligence
Published Date: May 1, 2024

Abstract

Purpose To evaluate the impact of an artificial intelligence (AI) assistant for lung cancer screening on multinational clinical workflows. Materials and Methods An AI assistant for lung cancer screening was evaluated on two retrospective randomized multireader multicase studies where 627 (141 cancer-positive cases) low-dose chest CT cases were each read twice (with and without AI assistance) by experienced thoracic radiologists (six U.S.-based or six Japan-based radiologists), resulting in a total of 7524 interpretations. Positive cases were defined as those within 2 years before a pathology-confirmed lung cancer diagnosis. Negative cases were defined as those without any subsequent cancer diagnosis for at least 2 years and were enriched for a spectrum of diverse nodules. The studies measured the readers' level of suspicion (on a 0-100 scale), country-specific screening system scoring categories, and management recommendations. Evaluation metrics included the area under the receiver operating characteristic curve (AUC) for level of suspicion and sensitivity and specificity of recall recommendations. Results With AI assistance, the radiologists' AUC increased by 0.023 (0.70 to 0.72; = .02) for the U.S. study and by 0.023 (0.93 to 0.96; = .18) for the Japan study. Scoring system specificity for actionable findings increased 5.5% (57% to 63%; < .001) for the U.S. study and 6.7% (23% to 30%; < .001) for the Japan study. There was no evidence of a difference in corresponding sensitivity between unassisted and AI-assisted reads for the U.S. (67.3% to 67.5%; = .88) and Japan (98% to 100%; > .99) studies. Corresponding stand-alone AI AUC system performance was 0.75 (95% CI: 0.70, 0.81) and 0.88 (95% CI: 0.78, 0.97) for the U.S.- and Japan-based datasets, respectively. Conclusion The concurrent AI interface improved lung cancer screening specificity in both U.S.- and Japan-based reader studies, meriting further study in additional international screening environments. Assistive Artificial Intelligence, Lung Cancer Screening, CT Published under a CC BY 4.0 license.

Authors

  • Atilla P Kiraly
    Google AI, Mountain View, CA, USA.
  • Corbin A Cunningham
    From Google Health Research, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (A.P.K., C.A.C., R.N., Z.N., C.L., J.R.L., D.A., S.M.M., R.P., Y.L., S.J., G.S.C., L.P., D.T., S.S., S.P., K.E.); Waymo, Mountain View, Calif (J.Y., N.B.), David Geffen School of Medicine at UCLA, Los Angeles, Calif (C.L.); Google, Mountain View, Calif (W.Y.); Department of Gastroenterology, Sendai Kousei Hospital, Sendai, Japan (H.S.); MNES Inc, Hiroshima, Japan (Y.S.); Department of Telemedicine, Northwestern University Feinberg School of Medicine, Chicago, Ill (M.E., D.M.); and Center for Biological Imaging, New York University-Langone Medical Center, New York, NY (D.P.N.).
  • Ryan Najafi
    From Google Health Research, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (A.P.K., C.A.C., R.N., Z.N., C.L., J.R.L., D.A., S.M.M., R.P., Y.L., S.J., G.S.C., L.P., D.T., S.S., S.P., K.E.); Waymo, Mountain View, Calif (J.Y., N.B.), David Geffen School of Medicine at UCLA, Los Angeles, Calif (C.L.); Google, Mountain View, Calif (W.Y.); Department of Gastroenterology, Sendai Kousei Hospital, Sendai, Japan (H.S.); MNES Inc, Hiroshima, Japan (Y.S.); Department of Telemedicine, Northwestern University Feinberg School of Medicine, Chicago, Ill (M.E., D.M.); and Center for Biological Imaging, New York University-Langone Medical Center, New York, NY (D.P.N.).
  • Zaid Nabulsi
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Jie Yang
    Key Laboratory of Development and Maternal and Child Diseases of Sichuan Province, Department of Pediatrics, Sichuan University, Chengdu, China.
  • Charles Lau
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Joseph R Ledsam
    DeepMind, London, UK.
  • Wenxing Ye
    Google AI, Mountain View, CA, USA.
  • Diego Ardila
    Google AI, Mountain View, CA, USA.
  • Scott M McKinney
    From Google Health Research, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (A.P.K., C.A.C., R.N., Z.N., C.L., J.R.L., D.A., S.M.M., R.P., Y.L., S.J., G.S.C., L.P., D.T., S.S., S.P., K.E.); Waymo, Mountain View, Calif (J.Y., N.B.), David Geffen School of Medicine at UCLA, Los Angeles, Calif (C.L.); Google, Mountain View, Calif (W.Y.); Department of Gastroenterology, Sendai Kousei Hospital, Sendai, Japan (H.S.); MNES Inc, Hiroshima, Japan (Y.S.); Department of Telemedicine, Northwestern University Feinberg School of Medicine, Chicago, Ill (M.E., D.M.); and Center for Biological Imaging, New York University-Langone Medical Center, New York, NY (D.P.N.).
  • Rory Pilgrim
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • Yun Liu
    Google Health, Palo Alto, CA USA.
  • Hiroaki Saito
    Department of Gastroenterology, Sendai Kousei Hospital, Miyagi, Japan.
  • Yasuteru Shimamura
    From Google Health Research, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (A.P.K., C.A.C., R.N., Z.N., C.L., J.R.L., D.A., S.M.M., R.P., Y.L., S.J., G.S.C., L.P., D.T., S.S., S.P., K.E.); Waymo, Mountain View, Calif (J.Y., N.B.), David Geffen School of Medicine at UCLA, Los Angeles, Calif (C.L.); Google, Mountain View, Calif (W.Y.); Department of Gastroenterology, Sendai Kousei Hospital, Sendai, Japan (H.S.); MNES Inc, Hiroshima, Japan (Y.S.); Department of Telemedicine, Northwestern University Feinberg School of Medicine, Chicago, Ill (M.E., D.M.); and Center for Biological Imaging, New York University-Langone Medical Center, New York, NY (D.P.N.).
  • Mozziyar Etemadi
    From the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta (O.T.I., M.B.P., A.Q.J., A.D., A.O.B.); Division of Cardiology (S.D., T.D.M., L.K.) and Department of Bioengineering and Therapeutic Sciences (S.R.), University of California, San Francisco; and Department of Anesthesiology and Department of Biomedical Engineering, Northwestern University, Chicago, IL (M.E., J.A.H.).
  • David Melnick
    Northwestern Medicine, Chicago, IL, USA.
  • Sunny Jansen
    Verily Life Sciences, South San Francisco, CA, USA.
  • Greg S Corrado
    Google Health, Palo Alto, CA USA.
  • Lily Peng
    Google Inc, Mountain View, California.
  • Daniel Tse
    Google AI, Mountain View, CA, USA. tsed@google.com.
  • Shravya Shetty
    Google AI, Mountain View, CA, USA.
  • Shruthi Prabhakara
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • David P Naidich
    New York University-Langone Medical Center, Center for Biological Imaging, New York City, NY, USA.
  • Neeral Beladia
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Krish Eswaran
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).

Keywords

External Resources

Popular Topics
Recent Journals