Performance of large language models in providing patient- oriented information on gingival recession: a comparative study.

Journal: The Saudi dental journal
Published Date:

Abstract

Patients are increasingly turning to large language models (LLMs) for health information, but the quality of patient-oriented information remains uncertain. This article evaluated the performance of three major LLMs in answering clinically relevant patient's questions related to gingival recession. A descriptive comparative study was carried out utilizing twenty standardized patient-centered questions related to gingival recession. Each question was submitted once to ChatGPT 5.5, Gemini 3.5, and Claude Sonnet 5, generating 60 responses. Five board-certified periodontists independently evaluated the responses using the validated Accuracy of Information (AOI) Index, a completeness scale, and expert preference rankings. Word count and response generation time were also recorded. Non-parametric tests and generalized estimating equations (GEE) were applied for statistical analysis, with statistical significance set at p ≤ 0.05. No significant differences were found among the three LLMs in accuracy (p = 0.993) or completeness (p = 0.972). ChatGPT generated responses significantly faster (median: 5.82 s) and with fewer words (median: 232) than Gemini (8.46 s; 438 words) and Claude (14.61 s; 333.5 words) (both p < 0.001). Expert preference rankings differed significantly (p = 0.002): Gemini and Claude were each ranked first in 38% of evaluations, compared with 24% for ChatGPT. For all three models, the highest accuracy was observed in Post-treatment Care. The three LLMs offered broadly comparable levels of accuracy and completeness when answering patient-oriented questions with respect to gingival recession. ChatGPT was faster and more concise, while Gemini and Claude were more often preferred by the expert evaluators. These LLMs may be useful as adjuncts to patient education, but they should complement rather than replace professional periodontal consultation.

Authors

Keywords

No keywords available for this article.