Proceedings From the 2022 ACR-RSNA Workshop on Safety, Effectiveness, Reliability, and Transparency in AI.

Journal: Journal of the American College of Radiology : JACR
Published Date: Feb 13, 2024

Abstract

Despite the surge in artificial intelligence (AI) development for health care applications, particularly for medical imaging applications, there has been limited adoption of such AI tools into clinical practice. During a 1-day workshop in November 2022, co-organized by the ACR and the RSNA, participants outlined experiences and problems with implementing AI in clinical practice, defined the needs of various stakeholders in the AI ecosystem, and elicited potential solutions and strategies related to the safety, effectiveness, reliability, and transparency of AI algorithms. Participants included radiologists from academic and community radiology practices, informatics leaders responsible for AI implementation, regulatory agency employees, and specialty society representatives. The major themes that emerged fell into two categories: (1) AI product development and (2) implementation of AI-based applications in clinical practice. In particular, participants highlighted key aspects of AI product development to include clear clinical task definitions; well-curated data from diverse geographic, economic, and health care settings; standards and mechanisms to monitor model reliability; and transparency regarding model performance, both in controlled and real-world settings. For implementation, participants emphasized the need for strong institutional governance; systematic evaluation, selection, and validation methods conducted by local teams; seamless integration into the clinical workflow; performance monitoring and support by local teams; performance monitoring by external entities; and alignment of incentives through credentialing and reimbursement. Participants predicted that clinical implementation of AI in radiology will continue to be limited until the safety, effectiveness, reliability, and transparency of such tools are more fully addressed.

Authors

  • David B Larson
    Department of Radiology, Warren Alpert Medical School, Brown University, 593 Eddy St, Providence, RI 02903 (I.P.); Department of Diagnostic Imaging, Rhode Island Hospital, Providence, RI (I.P.); Visiana, Hørsholm, Denmark (H.H.T.); Department of Radiology, Stanford University, Palo Alto, Calif (S.S.H., D.B.L.); and Department of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, Mass (J.K.C.).
  • Florence X Doo
    Department of Radiology, Mount Sinai Health System, New York, New York.
  • Bibb Allen
    Department of Radiology, Grandview Medical Center, Birmingham, Alabama. Electronic address: bibb@mac.com.
  • John Mongan
    From the Departments of Urology (T.C., M.U., H.C.C., M.S.) and Radiology and Biomedical Imaging (J.M., M.P.K., A.T., P.J., R.G., S.W.), University of California, San Francisco. 505 Parnassus Ave, M-391, San Francisco, CA 94143; and Division of Urology, Faculty of Medicine, King Chulalongkorn Memorial Hospital, Chulalongkorn University, The Thai Red Cross Society, Bangkok, Thailand (M.U.).
  • Adam E Flanders
  • Christoph Wald
    Chairman, Department of Radiology at Lahey Hospital & Medical Center, Professor of Radiology, Tufts University Medical School; Chair of the ACR Informatics Commission.

Keywords

External Resources

Popular Topics
Recent Journals