Deep Learning Detection of Active Pulmonary Tuberculosis at Chest Radiography Matched the Clinical Performance of Radiologists.

Journal: Radiology
Published Date: Sep 6, 2022

Abstract

Background The World Health Organization (WHO) recommends chest radiography to facilitate tuberculosis (TB) screening. However, chest radiograph interpretation expertise remains limited in many regions. Purpose To develop a deep learning system (DLS) to detect active pulmonary TB on chest radiographs and compare its performance to that of radiologists. Materials and Methods A DLS was trained and tested using retrospective chest radiographs (acquired between 1996 and 2020) from 10 countries. To improve generalization, large-scale chest radiograph pretraining, attention pooling, and semisupervised learning ("noisy-student") were incorporated. The DLS was evaluated in a four-country test set (China, India, the United States, and Zambia) and in a mining population in South Africa, with positive TB confirmed with microbiological tests or nucleic acid amplification testing (NAAT). The performance of the DLS was compared with that of 14 radiologists. The authors studied the efficacy of the DLS compared with that of nine radiologists using the Obuchowski-Rockette-Hillis procedure. Given WHO targets of 90% sensitivity and 70% specificity, the operating point of the DLS (0.45) was prespecified to favor sensitivity. Results A total of 165 754 images in 22 284 subjects (mean age, 45 years; 21% female) were used for model development and testing. In the four-country test set (1236 subjects, 17% with active TB), the receiver operating characteristic (ROC) curve of the DLS was higher than those for all nine India-based radiologists, with an area under the ROC curve of 0.89 (95% CI: 0.87, 0.91). Compared with these radiologists, at the prespecified operating point, the DLS sensitivity was higher (88% vs 75%, < .001) and specificity was noninferior (79% vs 84%, = .004). Trends were similar within other patient subgroups, in the South Africa data set, and across various TB-specific chest radiograph findings. In simulations, the use of the DLS to identify likely TB-positive chest radiographs for NAAT confirmation reduced the cost by 40%-80% per TB-positive patient detected. Conclusion A deep learning method was found to be noninferior to radiologists for the determination of active tuberculosis on digital chest radiographs. © RSNA, 2022 See also the editorial by van Ginneken in this issue.

Authors

  • Sahar Kazemzadeh
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • Jin Yu
    School of Biomedical Engineering and Imaging Sciences, King's College London, United Kingdom (C.H.R., I.S., J.Y., M.B., A.M., J.A.S.-L., I.K., J.W., C.C., O.R., M.O., S.E.W., S.A.N.).
  • Shahar Jamshy
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • Rory Pilgrim
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • Zaid Nabulsi
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Christina Chen
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Neeral Beladia
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Charles Lau
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Scott Mayer McKinney
    Google Health, Palo Alto, CA, USA. scottmayer@google.com.
  • Thad Hughes
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • Atilla P Kiraly
    Google AI, Mountain View, CA, USA.
  • Sreenivasa Raju Kalidindi
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Monde Muyoyeta
    Centre for Infectious Disease Research in Zambia, Lusaka, Zambia.
  • Jameson Malemela
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • Ting Shih
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).
  • Greg S Corrado
    Google Health, Palo Alto, CA USA.
  • Lily Peng
    Google Inc, Mountain View, California.
  • Katherine Chou
    Google Research, San Jose, CA, USA.
  • Po-Hsuan Cameron Chen
    Google Health, Palo Alto, CA USA.
  • Yun Liu
    Google Health, Palo Alto, CA USA.
  • Krish Eswaran
    From Google Health, Google, 3400 Hillview Ave, Palo Alto, CA 94304 (A.B.S., C.C., Z.N., Y. Liu, K.E., D.T., N.B., S.S.); Google Research, Cambridge, Mass (Y. Li, A.M., A.S., J.H., D.K.); Google via Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); and Northwestern Medicine, Chicago, Ill (M.E., F.G.V., D.M.).
  • Daniel Tse
    Google AI, Mountain View, CA, USA. tsed@google.com.
  • Shravya Shetty
    Google AI, Mountain View, CA, USA.
  • Shruthi Prabhakara
    From Google Health, 1600 Amphitheatre Pkwy, Mountain View, CA 94043 (S.K., J.Y., S.J., R.P., Z.N., C.C., N.B., S.M.M., T.H., A.P.K., G.S.C., L.P., K.C., P.H.C.C., Y.L., K.E., D.T., S.S., S.P.); Advanced Clinical, Deerfield, Ill (C.L.); Apollo Radiology International, Hyderabad, India (S.R.K.); TB Department, Center of Infectious Disease Research in Zambia, Lusaka, Zambia (M.M.); Sibanye Stillwater, Weltevreden Park, Roodepoort, South Africa (J.M.); and Clickmedix, Gaithersburg, Md (T.S.).

Keywords

External Resources

Popular Topics
Recent Journals