Accuracy and Reliability of ChatGPT in Answering Patient Questions About Lung Cancer and Its Surgery: An Expert Panel Evaluation by Thoracic Surgeons.
Journal:
Journal of cancer education : the official journal of the American Association for Cancer Education
Published Date:
Jul 4, 2025
Abstract
This study aimed to evaluate the accuracy, clarity, and scientific adequacy of ChatGPT's responses to frequently asked patient questions concerning lung cancer and its surgical treatment, through an expert panel of thoracic surgeons. A total of 36 frequently asked questions-20 related to lung cancer and 16 related to lung cancer surgery-were collected from various online sources and clinical experience. These questions were submitted to ChatGPT-4.0 in a single session, and the initial responses were assessed by four experienced thoracic surgeons. Each response was scored independently using a 5-point Likert scale for scientific adequacy, clarity, and accuracy. The mean scores, standard deviations, and word counts were calculated. Inter-group comparisons were conducted using independent-samples t-tests. ChatGPT's responses were rated generally high across all domains. For lung cancer questions, the mean scores were 4.50 ± 0.18 (scientific adequacy), 4.57 ± 0.21 (clarity), and 4.66 ± 0.21 (accuracy), with an average word count of 152.4 ± 36.86. For surgical questions, scores were slightly higher: 4.57 ± 0.31, 4.64 ± 0.26, and 4.73 ± 0.21, respectively, with an average word count of 163.68 ± 35.64. Although the differences were not statistically significant, responses to surgical questions were associated with slightly higher agreement scores. Full scores were achieved in three surgical questions. ChatGPT demonstrated a high degree of reliability and clarity in answering commonly asked patient questions about lung cancer and surgery. While the model can serve as a supportive educational tool, it should not replace personalized physician-patient communication, especially in clinical decision-making processes.
Authors
Keywords
No keywords available for this article.
