Assessing appropriate responses to ACR urologic imaging scenarios using ChatGPT and Bard.

Journal: Current problems in diagnostic radiology
Published Date: Oct 20, 2023

Abstract

Artificial intelligence (AI) has recently become a trending tool and topic regarding productivity especially with publicly available free services such as ChatGPT and Bard. In this report, we investigate if two widely available chatbots chatGPT and Bard, are able to show consistent accurate responses for the best imaging modality for urologic clinical situations and if they are in line with American College of Radiology (ACR) Appropriateness Criteria (AC). All clinical scenarios provided by the ACR were inputted into ChatGPT and Bard with result compared to the ACR AC and recorded. Both chatbots had an appropriate imaging modality rate of of 62% and no significant difference in proportion of correct imaging modality was found overall between the two services (p>0.05). The results of our study found that both ChatGPT and Bard are similar in their ability to suggest the most appropriate imaging modality in a variety of urologic scenarios based on ACR AC criteria. Nonetheless, both chatbots lack consistent accuracy and further development is necessary for implementation in clinical settings. For proper use of these AI services in clinical decision making, further developments are needed to improve the workflow of physicians.

Authors

  • Sishir Doddi
    University of Toledo College of Medicine, Toledo, OH, United States. Electronic address: sishir.doddi@utoledo.edu.
  • Taryn Hibshman
    University of Toledo College of Medicine, Toledo, OH, United States.
  • Oscar Salichs
    University of Toledo College of Medicine, Toledo, OH, United States.
  • Kaustav Bera
    Department of Biomedical Engineering, Case Western Reserve University School of Engineering, 2071 Martin Luther King Dr, Cleveland, OH 44106-7207 (M. Khorrami, K.B., A.M.); Departments of Internal Medicine (M. Khunger) and Solid Tumor Oncology (A.Z., P.P.), Cleveland Clinic, Cleveland, Ohio; Department of Internal Medicine, Maimonides Medical Center, Brooklyn, NY (R.T.); Department of Radiology, UT Southwestern Medical Center, Dallas, Tex (P.R.); Department of Population and Quantitative Health Sciences, Case Western Reserve University, Cleveland, Ohio (P.F.); Department of Hematology and Oncology, New York University, New York, NY (V.V.); Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, Ohio (A.M.).
  • Charit Tippareddy
    Department of Radiology, University Hospitals Cleveland Medical Center, Cleveland, OH.
  • Nikhil Ramaiya
    Department of Radiology, University Hospitals Cleveland Medical Center, Case Western Reserve University, Cleveland, Ohio.
  • Sree Harsha Tirumani
    Department of Radiology, University Hospitals Cleveland Medical Center, Cleveland, OH, USA.

Keywords

External Resources

Popular Topics
Recent Journals