Improving Radiographic Fracture Recognition Performance and Efficiency Using Artificial Intelligence.

Journal: Radiology
PMID: 34931859

Abstract

Background Missed fractures are a common cause of diagnostic discrepancy between initial radiographic interpretation and the final read by board-certified radiologists. Purpose To assess the effect of assistance by artificial intelligence (AI) on diagnostic performances of physicians for fractures on radiographs. Materials and Methods This retrospective diagnostic study used the multi-reader, multi-case methodology based on an external multicenter data set of 480 examinations with at least 60 examinations per body region (foot and ankle, knee and leg, hip and pelvis, hand and wrist, elbow and arm, shoulder and clavicle, rib cage, and thoracolumbar spine) between July 2020 and January 2021. Fracture prevalence was set at 50%. The ground truth was determined by two musculoskeletal radiologists, with discrepancies solved by a third. Twenty-four readers (radiologists, orthopedists, emergency physicians, physician assistants, rheumatologists, family physicians) were presented the whole validation data set ( = 480), with and without AI assistance, with a 1-month minimum washout period. The primary analysis had to demonstrate superiority of sensitivity per patient and the noninferiority of specificity per patient at -3% margin with AI aid. Stand-alone AI performance was also assessed using receiver operating characteristic curves. Results A total of 480 patients were included (mean age, 59 years ± 16 [standard deviation]; 327 women). The sensitivity per patient was 10.4% higher (95% CI: 6.9, 13.9; < .001 for superiority) with AI aid (4331 of 5760 readings, 75.2%) than without AI (3732 of 5760 readings, 64.8%). The specificity per patient with AI aid (5504 of 5760 readings, 95.6%) was noninferior to that without AI aid (5217 of 5760 readings, 90.6%), with a difference of +5.0% (95% CI: +2.0, +8.0; = .001 for noninferiority). AI shortened the average reading time by 6.3 seconds per examination (95% CI: -12.5, -0.1; = .046). The sensitivity by patient gain was significant in all regions (+8.0% to +16.2%; < .05) but shoulder and clavicle and spine (+4.2% and +2.6%; = .12 and .52). Conclusion AI assistance improved the sensitivity and may even improve the specificity of fracture detection by radiologists and nonradiologists, without lengthening reading time. Published under a CC BY 4.0 license. See also the editorial by Link and Pedoia in this issue.

Authors

  • Ali Guermazi
    Department of Radiology, Boston University School of Medicine, Boston, MA, 02118, USA.
  • Chadi Tannoury
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Andrew J Kompel
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Akira M Murakami
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Alexis Ducarouge
    Gleamer, Paris, France.
  • André Gillibert
    Department of Epidemiology and Public Health, Rouen University Hospital, Rouen, France.
  • Xinning Li
    Department of Orthopaedic Surgery, Boston University School of Medicine, Boston, MA, USA.
  • Antoine Tournier
    Gleamer, Paris, France.
  • Youmna Lahoud
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Mohamed Jarraya
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Elise Lacave
    From the Department of Radiology, Hôpital Fondation A. de Rothschild, 25 rue Manin, 75019 Paris, France (L.D.); Faculty of Medicine, Université de Paris, Paris, France (L.D., A. Feydy); Gleamer, Paris, France (A.D., C.A., N.C., Z.Z., N.N., E.L., A.P., N.E.R.); Department of Biostatistics, CHU Rouen, Rouen, France (A.G.); Department of Radiology, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, Paris, France (J.L.); Department of Radiology, Hôpital Ambroise-Paré, Assistance Publique-Hôpitaux de Paris, Boulogne-Billancourt, France (A. Felter); Department of Radiology, Hôpital Raymond-Poincaré, Assistance Publique-Hôpitaux de Paris, Garches, France (A. Felter); and Department of Radiology B, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Paris, France (L.L., N.E.R., A. Feydy).
  • Hamza Rahimi
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Aloïs Pourchot
    From the Department of Radiology, Hôpital Fondation A. de Rothschild, 25 rue Manin, 75019 Paris, France (L.D.); Faculty of Medicine, Université de Paris, Paris, France (L.D., A. Feydy); Gleamer, Paris, France (A.D., C.A., N.C., Z.Z., N.N., E.L., A.P., N.E.R.); Department of Biostatistics, CHU Rouen, Rouen, France (A.G.); Department of Radiology, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, Paris, France (J.L.); Department of Radiology, Hôpital Ambroise-Paré, Assistance Publique-Hôpitaux de Paris, Boulogne-Billancourt, France (A. Felter); Department of Radiology, Hôpital Raymond-Poincaré, Assistance Publique-Hôpitaux de Paris, Garches, France (A. Felter); and Department of Radiology B, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Paris, France (L.L., N.E.R., A. Feydy).
  • Robert L Parisien
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Alexander C Merritt
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Douglas Comeau
    From the Departments of Radiology (A. Guermazi, A.J.K., A.M.M., H.R., A.C.M., D.H.), Orthopaedic Surgery (C.T., X.L.), and Family Medicine (D.C.), Boston University School of Medicine, Boston, Mass; Department of Radiology, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A. Guermazi); Gleamer, Paris, France (A.D., A.T., E.L., A.P., N.E.); Department of Biostatistics, CHU Rouen, Rouen, France (A. Gillibert); Department of Rheumatology, Harvard Vanguard Medical Associates, Braintree, Mass (Y.L.); Department of Radiology, Musculoskeletal Division, Massachusetts General Hospital, Harvard Medical School, Boston, Mass 02114 (M.J.); Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, Paris, France (A.P.); Department of Orthopaedic Surgery, The Mount Sinai Hospital, New York, NY (R.L.P.); University Health Services and Primary Care Sports Medicine, Boston College, Chestnut Hill, Mass (D.C.); and Department of Radiology, Stony Brook University Renaissance School of Medicine, Stony Brook, NY (D.H.).
  • Nor-Eddine Regnard
    From the Department of Radiology, Hôpital Fondation A. de Rothschild, 25 rue Manin, 75019 Paris, France (L.D.); Faculty of Medicine, Université de Paris, Paris, France (L.D., A. Feydy); Gleamer, Paris, France (A.D., C.A., N.C., Z.Z., N.N., E.L., A.P., N.E.R.); Department of Biostatistics, CHU Rouen, Rouen, France (A.G.); Department of Radiology, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, Paris, France (J.L.); Department of Radiology, Hôpital Ambroise-Paré, Assistance Publique-Hôpitaux de Paris, Boulogne-Billancourt, France (A. Felter); Department of Radiology, Hôpital Raymond-Poincaré, Assistance Publique-Hôpitaux de Paris, Garches, France (A. Felter); and Department of Radiology B, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Paris, France (L.L., N.E.R., A. Feydy).
  • Daichi Hayashi
    Department of Radiology, Stony Brook Medicine, State University of New York, United States; Department of Radiology, Boston University School of Medicine, Boston, United States. Electronic address: Daichi.Hayashi@stonybrookmedicine.edu.

Keywords

External Resources

Popular Topics
Recent Journals