Evaluating the concordance of ChatGPT and physician recommendations for bariatric surgery.

Journal: Canadian journal of physiology and pharmacology
PMID: 39561352

Abstract

Integrating artificial intelligence (AI) into healthcare prompts the need to measure its proficiency relative to human experts. This study evaluates the proficiency of ChatGPT, an OpenAI language model, in offering guidance concerning bariatric surgery compared to bariatric surgeons. Five clinical scenarios representative of diverse bariatric surgery situations were given to American Society for Metabolic and Bariatric Surgery (ASMBS)-accredited bariatric surgeons and ChatGPT. Both groups proposed medical or surgical management for the patients depicted in each scenario. The outcomes from both the surgeons and ChatGPT were examined and matched with the clinical benchmarks set by the ASMBS. There was a high degree of agreement between ChatGPT and physicians on the three simpler clinical scenarios. There was a positive correlation between physicians' and ChatGPT answers for not recommending surgery. ChatGPT's advice aligned with ASMBS guidelines 60% of the time, in contrast to bariatric surgeons, who consistently aligned with the guidelines 100% of the time. ChatGPT showcases potential in offering guidance on bariatric surgery, but it does not have the comprehensive and personalized perspective that doctors exhibit consistently. Enhancing AI's training on intricate patient situations will bolster its role in the medical field.

Authors

  • Sunny Kahlon
    University of South Florida Health Morsani College of Medicine, Tampa, FL, USA.
  • Mary Sleet
    University of South Florida Health Morsani College of Medicine, Tampa, FL, USA.
  • Joseph Sujka
    Department of Surgery, University of South Florida Morsani College of Medicine, Tampa, FL, USA.
  • Salvatore Docimo
    Division of Bariatric, Foregut and Advanced Gastrointestinal Surgery, Department of Surgery, Stony Brook University Medical Center, 100 Nichols Road, HSC T19, Stony Brook, NY, 11794, USA.
  • Christopher DuCoin
    Department of Surgery, USF Morsani College of Medicine, University of South Florida, 12901 Bruce B Downs Blvd, MDC 3129, Tampa, FL, 33612, USA.
  • Francesca Dimou
    Department of Surgery, University of South Florida Morsani College of Medicine, Tampa, FL, USA.
  • Rahul Mhaskar
    Department of Internal Medicine, University of South Florida Morsani College of Medicine, Tampa, FL, USA.

Keywords

External Resources

Popular Topics
Recent Journals