Accuracy and Reliability of Chatbot Responses to Physician Questions.

Journal: JAMA network open
Published Date: Oct 2, 2023

Abstract

IMPORTANCE: Natural language processing tools, such as ChatGPT (generative pretrained transformer, hereafter referred to as chatbot), have the potential to radically enhance the accessibility of medical information for health professionals and patients. Assessing the safety and efficacy of these tools in answering physician-generated questions is critical to determining their suitability in clinical settings, facilitating complex decision-making, and optimizing health care efficiency.

Authors

  • Rachel S Goodman
    Vanderbilt University School of Medicine, Nashville, TN, USA.
  • J Randall Patrinely
    Department of Dermatology, Vanderbilt University Medical Center, Nashville, TN, USA.
  • Cosby A Stone
    Department of Allergy, Pulmonology, and Critical Care, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Eli Zimmerman
    Department of Neurology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Rebecca R Donald
    Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Sam S Chang
  • Sean T Berkowitz
    Vanderbilt Eye Institute, Department of Ophthalmology, Vanderbilt University Medical, Nashville, Tennessee.
  • Avni P Finn
    Vanderbilt Eye Institute, Department of Ophthalmology, Vanderbilt University Medical, Nashville, Tennessee.
  • Eiman Jahangir
    Department of Cardiovascular Medicine, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Elizabeth A Scoville
    Department of Gastroenterology, Hepatology, and Nutrition, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Tyler S Reese
    Department of Rheumatology and Immunology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Debra L Friedman
    Department of Pediatric Hematology/Oncology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Julie A Bastarache
    Department of Allergy, Pulmonology, and Critical Care, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Yuri F van der Heijden
    Department of Infectious Disease, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Jordan J Wright
    Department of Diabetes, Endocrinology, and Metabolism, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Fei Ye
    School of information science and technology, Southwest Jiaotong University, ChengDu, China.
  • Nicholas Carter
    Division of Trauma and Surgical Critical Care, University of Miami Miller School of Medicine, Miami, Florida.
  • Matthew R Alexander
    Department of Cardiovascular Medicine and Clinical Pharmacology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Jennifer H Choe
    Department of Hematology/Oncology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Cody A Chastain
    Department of Infectious Disease, Vanderbilt University Medical Center, Nashville, Tennessee.
  • John A Zic
    Department of Dermatology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Sara N Horst
    Department of Gastroenterology, Hepatology, and Nutrition, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Isik Turker
    Department of Cardiology, Washington University School of Medicine in St Louis, St Louis, Missouri.
  • Rajiv Agarwal
    Department of Medicine (R.A., J.G.W., M.B.K.), Vanderbilt University Medical Center, Nashville, Tennessee, USA; Vanderbilt-Ingram Cancer Center (R.A., J.G.W.), Nashville, Tennessee, USA. Electronic address: rajiv.agarwal@vumc.org.
  • Evan Osmundson
    Department of Radiation Oncology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Kamran Idrees
    Department of Surgical Oncology & Endocrine Surgery, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Colleen M Kiernan
    Department of Surgical Oncology, The University of Texas MD Anderson Cancer Center, 1400 Pressler St, Unit 1484, PO Box 301402, Houston, TX, 77230, USA.
  • Chandrasekhar Padmanabhan
    Department of Surgical Oncology & Endocrine Surgery, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Christina E Bailey
    Department of Surgical Oncology & Endocrine Surgery, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Cameron E Schlegel
    Department of Surgical Oncology & Endocrine Surgery, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Lola B Chambless
    Department of Neurosurgery, Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee, United States.
  • Michael K Gibson
    Department of Hematology/Oncology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Travis J Osterman
    Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Lee E Wheless
    Department of Dermatology, Vanderbilt University Medical Center, Nashville, Tennessee.
  • Douglas B Johnson
    Vanderbilt-Ingram Cancer Center, Vanderbilt University Medical Center, Nashville, TN, USA.

Keywords

External Resources

Popular Topics
Recent Journals