Semantic Clinical Artificial Intelligence vs Native Large Language Model Performance on the USMLE.

Journal: JAMA network open
PMID:

Abstract

IMPORTANCE: Large language models (LLMs) are being implemented in health care. Enhanced accuracy and methods to maintain accuracy over time are needed to maximize LLM benefits.

Authors

  • Peter L Elkin
    Department of Biomedical Informatics, University at Buffalo, Buffalo, NY.
  • Guresh Mehta
    Department of Biomedical Informatics, University at Buffalo.
  • Frank LeHouillier
    Department of Biomedical Informatics, University at Buffalo.
  • Melissa Resnick
    University at Buffalo, Department of Biomedical Informatics, Buffalo, New York USA.
  • Sarah Mullin
    University at Buffalo, The State University of New York, USA.
  • Crystal Tomlin
    Department of Biomedical Informatics, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, New York.
  • Skyler Resendez
    Department of Biomedical Informatics, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, New York.
  • Jiaxing Liu
    School of Statistics and Mathematics, Zhongnan University of Economics and Law, Wuhan, China.
  • Jonathan R Nebeker
    Department of Veterans Affairs, Office of Health Informatics, USA.
  • Steven H Brown
    Office of Health Informatics, Department of Veterans Affairs.