Evaluating ChatGPT's Utility in Medicine Guidelines Through Web Search Analysis.

Journal: The Permanente journal
PMID:

Abstract

INTRODUCTION: With the rise of machine learning applications in health care, shifts in medical fields that rely on precise prognostic models and pattern detection tools are anticipated in the near future. Chat Generative Pretrained Transformer (ChatGPT) is a recent machine learning innovation known for producing text that mimics human conversation. To gauge ChatGPT's capability in addressing patient inquiries, the authors set out to juxtapose it with Google Search, America's predominant search engine. Their comparison focused on: 1) the top questions related to clinical practice guidelines from the American Academy of Family Physicians by category and subject; 2) responses to these prevalent questions; and 3) the top questions that elicited a numerical reply.

Authors

  • Jeremy A Dubin
    LifeBridge Health, Sinai Hospital of Baltimore, Rubin Institute for Advanced Orthopedics, Baltimore, Maryland.
  • Sandeep S Bains
    LifeBridge Health, Sinai Hospital of Baltimore, Rubin Institute for Advanced Orthopedics, Baltimore, Maryland.
  • Daniel Hameed
    LifeBridge Health, Sinai Hospital of Baltimore, Rubin Institute for Advanced Orthopedics, Baltimore, Maryland.
  • Zhongming Chen
    LifeBridge Health, Sinai Hospital of Baltimore, Rubin Institute for Advanced Orthopedics, Baltimore, Maryland.
  • Erica Gaertner
    Department of Family Medicine, LifeBridge Health, Sinai Hospital, Baltimore, MD, USA.
  • James Nace
    Rubin Institute for Advanced Orthopedics, Sinai Hospital of Baltimore, Baltimore, Maryland.
  • Michael A Mont
    Department of Orthopaedic Surgery, Lenox Hill Hospital of Northwell Health, New York, NY.
  • Ronald E Delanois
    Rubin Institute for Advanced Orthopedics, Sinai Hospital of Baltimore, Baltimore, Maryland.