Examining ChatGPT Performance on USMLE Sample Items and Implications for Assessment.
Journal:
Academic medicine : journal of the Association of American Medical Colleges
Published Date:
Nov 7, 2023
Abstract
PURPOSE: In late 2022 and early 2023, reports that ChatGPT could pass the United States Medical Licensing Examination (USMLE) generated considerable excitement, and media response suggested ChatGPT has credible medical knowledge. This report analyzes the extent to which an artificial intelligence (AI) agent's performance on these sample items can generalize to performance on an actual USMLE examination and an illustration is given using ChatGPT.