Comparative Accuracy of Generative Artificial Intelligence Platforms on Predoctoral Pediatric Dentistry Examination.
Journal:
Pediatric dentistry
PMID:
40296265
Abstract
To determine the comparative accuracy of seven generative artificial intelligence (GenAI) platforms in answering multiple-choice questions on a predoctoral pediatric dentistry examination. This study evaluated the impact of question type and GenAI training on accuracy. In this study, 100 multiple-choice questions were answered by seven GenAIs using a standard prompt. The study included five untrained GenAIs (Llama, Gemini, Copilot, ChatGPT3.5, and ChatGPT4) and two trained GenAIs (ChatGPT3.5 and ChatGPT4). The training of GenAIs was performed using evidence-based data. The questions were categorized as knowledge-based versus critical thinking on 10 subspecialty domains. The GenAIs were asked to select one correct answer from four choices, and only the first generated response was recorded. Data were subjected to statistical analysis (alpha equals 0.05), with a passing score of 75 percent. Trained ChatGPT4 had the highest accuracy score (90 percent), while untrained Copilot had the lowest accuracy score (57 percent). Only three GenAIs received a passing score (trained ChatGPT3.5, untrained and trained ChatGPT4). The average performance of these three GenAIs (87 percent) was comparable to that of dental students (89 percent). There was no difference in the accuracy of GenAI in answering knowledge-based or critical-thinking questions. Similarly, sub-specialty domain types did not impact the accuracy of GenAI. Newer or trained models of generative artificial intelligence have higher accuracy compared to older or untrained models of GenAI. In the future, due to high accuracy, newer or trained models of GenAI can be used as adjuncts in predoctoral pediatric dental education.