Comparative evaluation of ChatGPT and Gemini in brain-computer interfaces patient education: A multi-dimensional analysis of reliability, accuracy, comprehensibility, and readability.

BACKGROUND: Brain-Computer Interfaces (BCI) are a type of life-altering neurotechnology, but their inherent complexity poses significant challenges to patient education. Large Language Models (LLMs), such as ChatGPT and Gemini, offer new possibilities to address this challenge. This study aims to conduct a multi-dimensional, rigorous comparative analysis of the performance of these two mainstream AI models in responding to common patient questions related to BCI. METHODS: Through a structured process combining clinical expert consensus, literature review, and online patient community analysis, we identified 13 key patient questions covering the entire BCI treatment cycle. We then obtained responses to these questions from ChatGPT and Gemini on September 1, 2025. An evaluation panel, composed of clinical experts and non-medical professionals, conducted a blinded assessment of the response quality using standardized Likert scales across three dimensions: reliability, accuracy, and comprehensibility. Concurrently, we performed an objective, quantitative analysis of the response texts using the Flesch-Kincaid readability tests. RESULTS: On core quality metrics such as reliability, accuracy, and comprehensibility, the performance of the two models was generally comparable, both demonstrating a high level of proficiency with only sporadic statistical differences on a few technical questions. However, a clear significant disparity emerged in the dimension of readability: for 12 of the 13 questions, the text generated by Gemini required a significantly lower reading grade level than that of ChatGPT (p < 0.05) and had significantly higher reading ease scores. This difference stemmed from Gemini's tendency to use shorter sentences and simpler vocabulary. CONCLUSION: AI chatbots possess immense potential in the field of BCI patient education. Although both ChatGPT and Gemini can provide high-quality information, Gemini demonstrates a clear advantage in the accessibility and approachability of information, making it a potentially more suitable tool for initial application across diverse patient populations. Nevertheless, the limitations of AI in handling highly specialized and dynamically changing knowledge underscore the indispensable role of human expert supervision and validation in any clinical application.