Improved Productivity Using Deep Learning-assisted Reporting for Lumbar Spine MRI.

Journal: Radiology
PMID: 35699577

Abstract

Background Lumbar spine MRI studies are widely used for back pain assessment. Interpretation involves grading lumbar spinal stenosis, which is repetitive and time consuming. Deep learning (DL) could provide faster and more consistent interpretation. Purpose To assess the speed and interobserver agreement of radiologists for reporting lumbar spinal stenosis with and without DL assistance. Materials and Methods In this retrospective study, a DL model designed to assist radiologists in the interpretation of spinal canal, lateral recess, and neural foraminal stenoses on lumbar spine MRI scans was used. Randomly selected lumbar spine MRI studies obtained in patients with back pain who were 18 years and older over a 3-year period, from September 2015 to September 2018, were included in an internal test data set. Studies with instrumentation and scoliosis were excluded. Eight radiologists, each with 2-13 years of experience in spine MRI interpretation, reviewed studies with and without DL model assistance with a 1-month washout period. Time to diagnosis (in seconds) and interobserver agreement (using Gwet κ) were assessed for stenosis grading for each radiologist with and without the DL model and compared with test data set labels provided by an external musculoskeletal radiologist (with 32 years of experience) as the reference standard. Results Overall, 444 images in 25 patients (mean age, 51 years ± 20 [SD]; 14 women) were evaluated in a test data set. DL-assisted radiologists had a reduced interpretation time per spine MRI study, from a mean of 124-274 seconds (SD, 25-88 seconds) to 47-71 seconds (SD, 24-29 seconds) ( < .001). DL-assisted radiologists had either superior or equivalent interobserver agreement for all stenosis gradings compared with unassisted radiologists. DL-assisted general and in-training radiologists improved their interobserver agreement for four-class neural foraminal stenosis, with κ values of 0.71 and 0.70 (with DL) versus 0.39 and 0.39 (without DL), respectively (both < .001). Conclusion Radiologists who were assisted by deep learning for interpretation of lumbar spinal stenosis on MRI scans showed a marked reduction in reporting time and superior or equivalent interobserver agreement for all stenosis gradings compared with radiologists who were unassisted by deep learning. © RSNA, 2022 . See also the editorial by Hayashi in this issue.

Authors

  • Desmond Shi Wei Lim
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Andrew Makmur
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Lei Zhu
    School of Civil and Hydraulic Engineering, Ningxia University, Yinchuan, China.
  • Wenqiao Zhang
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Amanda J L Cheng
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • David Soon Yiew Sia
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Sterling Ellis Eide
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Han Yang Ong
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Pooja Jagmohan
    Department of Diagnostic Imaging, National University Hospital, Singapore.
  • Wei Chuan Tan
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Vanessa Meihui Khoo
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Ying Mei Wong
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Yee Liang Thian
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Sangeetha Baskar
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Ee Chin Teo
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Diyaa Abdul Rauf Algazwi
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Qai Ven Yap
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Yiong Huak Chan
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Jiong Hao Tan
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Naresh Kumar
    Department of Computer Science & Engineering, Maharaja Surajmal Institute of Technology, C-4, Janakpuri, New Delhi 110058, India.
  • Beng Chin Ooi
    School of Computing, National University of Singapore, Singapore, Singapore.
  • Hiroshi Yoshioka
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • Swee Tian Quek
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).
  • James Thomas Patrick Decourcy Hallinan
    From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.).

Keywords

External Resources

Popular Topics
Recent Journals