RSNA 2023 Abdominal Trauma AI Challenge: Review and Outcomes.

Journal: Radiology. Artificial intelligence
PMID: 39503604

Abstract

Purpose To evaluate the performance of the winning machine learning models from the 2023 RSNA Abdominal Trauma Detection AI Challenge. Materials and Methods The competition was hosted on Kaggle and took place between July 26 and October 15, 2023. The multicenter competition dataset consisted of 4274 abdominal trauma CT scans, in which solid organs (liver, spleen, and kidneys) were annotated as healthy, low-grade, or high-grade injury. Studies were labeled as positive or negative for the presence of bowel and mesenteric injury and active extravasation. In this study, performances of the eight award-winning models were retrospectively assessed and compared using various metrics, including the area under the receiver operating characteristic curve (AUC), for each injury category. The reported mean values of these metrics were calculated by averaging the performance across all models for each specified injury type. Results The models exhibited strong performance in detecting solid organ injuries, particularly high-grade injuries. For binary detection of injuries, the models demonstrated mean AUC values of 0.92 (range, 0.90-0.94) for liver, 0.91 (range, 0.87-0.93) for splenic, and 0.94 (range, 0.93-0.95) for kidney injuries. The models achieved mean AUC values of 0.98 (range, 0.96-0.98) for high-grade liver, 0.98 (range, 0.97-0.99) for high-grade splenic, and 0.98 (range, 0.97-0.98) for high-grade kidney injuries. For the detection of bowel and mesenteric injuries and active extravasation, the models demonstrated mean AUC values of 0.85 (range, 0.74-0.93) and 0.85 (range, 0.79-0.89), respectively. Conclusion The award-winning models from the artificial intelligence challenge demonstrated strong performance in the detection of traumatic abdominal injuries on CT scans, particularly high-grade injuries. These models may serve as a performance baseline for future investigations and algorithms. Abdominal Trauma, CT, American Association for the Surgery of Trauma, Machine Learning, Artificial Intelligence © RSNA, 2024.

Authors

  • Sebastiaan Hermans
    Department of Medical Imaging, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.
  • Zixuan Hu
    The Edward S. Rogers Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada.
  • Robyn L Ball
    From the Department of Radiology, Stanford University School of Medicine, Stanford University Medical Center, 725 Welch Rd, Room 1675, Stanford, Calif 94305-5913 (M.C.C., N.M., D.B.L., C.P.L., M.P.L.); Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, Calif (R.L.B., L.Y.); Department of Bioinformatics, University of Utah Medical Center, Salt Lake City, Utah (B.E.C.); and Department of Radiology, Duke University Medical Center, Durham, NC (T.J.A.).
  • Hui Ming Lin
    Department of Medical Imaging, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.
  • Luciano M Prevedello
  • Ferco H Berger
    From the Department of Medical Imaging, St Michael's Hospital, Unity Health Toronto, 30 Bond St, Toronto, ON, Canada M5B 1W8 (S.H., Z.H., H.M.L., I.Y., E.C.); Edward S. Rogers Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada (Z.H., E.S.); The Jackson Laboratory, Bar Harbor, Me (R.L.B.); Department of Radiology, The Ohio State University, Columbus, Ohio (L.M.P.); Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada (F.H.B.); Department of Radiology, Scripps Clinic Medical Group and University of California San Diego, San Diego, Calif (J.D.R.); Radiological Society of North America, Oak Brook, Ill (M.V.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa (A.E.F.); Department of Radiology, Weill Cornell Medicine, New York, NY (G.S.); Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, Calif (J.M.), Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Memorial Sloan-Kettering Cancer Center, New York, NY (B.S.M.); Department of Radiology and Biomedical Imaging, Yale University School of Medicine, New Haven, Conn (M.A.D.); Duke University School of Medicine, Durham, NC (K.M.); North York General Hospital, Toronto, Ontario, Canada (E.S.); and Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada (E.C.).
  • Ibrahim Yusuf
    From the Department of Medical Imaging, St Michael's Hospital, Unity Health Toronto, 30 Bond St, Toronto, ON, Canada M5B 1W8 (S.H., Z.H., H.M.L., I.Y., E.C.); Edward S. Rogers Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada (Z.H., E.S.); The Jackson Laboratory, Bar Harbor, Me (R.L.B.); Department of Radiology, The Ohio State University, Columbus, Ohio (L.M.P.); Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada (F.H.B.); Department of Radiology, Scripps Clinic Medical Group and University of California San Diego, San Diego, Calif (J.D.R.); Radiological Society of North America, Oak Brook, Ill (M.V.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa (A.E.F.); Department of Radiology, Weill Cornell Medicine, New York, NY (G.S.); Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, Calif (J.M.), Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Memorial Sloan-Kettering Cancer Center, New York, NY (B.S.M.); Department of Radiology and Biomedical Imaging, Yale University School of Medicine, New Haven, Conn (M.A.D.); Duke University School of Medicine, Durham, NC (K.M.); North York General Hospital, Toronto, Ontario, Canada (E.S.); and Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada (E.C.).
  • Jeffrey D Rudie
    Department of Radiology, Hospital of the University of Pennsylvania, 3400 Spruce St, Philadelphia, PA 19104 (J.D.R., L.X., A.K., J.M.E., T.C., I.M.N., S.M., J.C.G.); Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, Calif (J.D.R., A.M.R.); Penn Image Computing and Science Laboratory, University of Pennsylvania, Philadelphia, Pa (X.L., J.W.); University of Pennsylvania Perelman School of Medicine, Philadelphia, Pa (M.T.D.); Mecklenburg Radiology Associates, Charlotte, NC (E.J.B.); Department of Radiology, University of Texas, Austin, Tex (R.N.B.); and Division of Nuclear Medicine and Clinical Molecular Imaging, Department of Radiology, University of Pennsylvania, Philadelphia, Pa (I.M.N.).
  • Maryam Vazirabad
    From the Department of Applied Innovation and AI, Dasa, São Paulo, Brazil (F.C.K.); Department of Diagnostic Imaging, Universidade Federal de São Paulo (Unifesp), Av Prof Ascendino Reis, 1245, 131, São Paulo, SP, Brazil 04027-000 (F.C.K.); Department of Radiology, The Ohio State University Wexner Medical Center, Columbus, Ohio (L.M.P.); Department of Medical Imaging, University of Toronto, Toronto, Canada (E.C.); Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, Ill (S.S.H.); Microsoft HLS, Redmond, Wash (M.P.L.); Department of Biomedical Data Science, Stanford University, Stanford, Calif (M.P.L.); The Jackson Laboratory, Bar Harbor, Maine (R.L.B.); Department of Ophthalmology, University of Colorado Denver School of Medicine, Aurora, Colo (J.K.C.); Department of Radiology, University of Pennsylvania, Philadelphia, Pa (C.E.K.); Department of Radiology, University of Utah, Salt Lake City, Utah (T.R.); Department of Radiology and Biomedical Imaging (M.P.L., J.F.T., J.M.) and Center for Intelligent Imaging (J.M.), University of California San Francisco, San Francisco, Calif; Department of Radiology, Weill Cornell Medical College, New York, NY (G.S.); Department of Medical Imaging, Unity Health Toronto, Toronto, Canada (H.M.L.); Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, MGB Data Science Office, Boston, Mass (K.P.A.); Informatics Department, Radiological Society of North America, Oak Brook, Ill (M.V.); Department of Radiology, Mayo Clinic, Rochester, Minn (B.J.E.); and Department of Radiology, Thomas Jefferson University, Philadelphia, Pa (A.E.F.).
  • Adam E Flanders
  • George Shih
  • John Mongan
    From the Departments of Urology (T.C., M.U., H.C.C., M.S.) and Radiology and Biomedical Imaging (J.M., M.P.K., A.T., P.J., R.G., S.W.), University of California, San Francisco. 505 Parnassus Ave, M-391, San Francisco, CA 94143; and Division of Urology, Faculty of Medicine, King Chulalongkorn Memorial Hospital, Chulalongkorn University, The Thai Red Cross Society, Bangkok, Thailand (M.U.).
  • Savvas Nicolaou
    Department of Radiology, Vancouver General Hospital, University of British Columbia, Vancouver, 899 12th Avenue West, British Columbia V5Z 1M9, Canada. Electronic address: savvas.nicolaou@vch.ca.
  • Brett S Marinelli
    From the Department of Medical Imaging, St Michael's Hospital, Unity Health Toronto, 30 Bond St, Toronto, ON, Canada M5B 1W8 (S.H., Z.H., H.M.L., I.Y., E.C.); Edward S. Rogers Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada (Z.H., E.S.); The Jackson Laboratory, Bar Harbor, Me (R.L.B.); Department of Radiology, The Ohio State University, Columbus, Ohio (L.M.P.); Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada (F.H.B.); Department of Radiology, Scripps Clinic Medical Group and University of California San Diego, San Diego, Calif (J.D.R.); Radiological Society of North America, Oak Brook, Ill (M.V.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa (A.E.F.); Department of Radiology, Weill Cornell Medicine, New York, NY (G.S.); Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, Calif (J.M.), Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Memorial Sloan-Kettering Cancer Center, New York, NY (B.S.M.); Department of Radiology and Biomedical Imaging, Yale University School of Medicine, New Haven, Conn (M.A.D.); Duke University School of Medicine, Durham, NC (K.M.); North York General Hospital, Toronto, Ontario, Canada (E.S.); and Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada (E.C.).
  • Melissa A Davis
    Department of Radiology and Biomedical Imaging, Yale University School of Medicine, 330 Cedar St. Tompkins East TE-2, New Haven, CT 06520.
  • Kirti Magudia
    Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, Mass (W.F.W., M.T.C., K.M., S.A.G., E.G., M.H.R., G.C.G., K.P.A.); and MGH & BWH Center for Clinical Data Science, Boston, Mass (W.F.W., M.T.C., K.M., K.P.A.).
  • Ervin Sejdić
    Department of Electrical and Computer Engineering, University of Pittsburgh, Benedum Hall, Pittsburgh, PA 15260, USA. Electronic address: esejdic@ieee.org.
  • Errol Colak
    Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada.

Keywords

External Resources

Popular Topics
Recent Journals