Readability, reliability, and quality of AI-Generated responses on pelvic congestion syndrome: A comparative analysis of ChatGPT, gemini, and perplexity.

ObjectivePelvic congestion syndrome (PCS) is a vascular cause of chronic pelvic pain arising from ovarian and pelvic venous insufficiency, typically presenting as dull, aching pelvic pain that worsens with prolonged standing, physical activity, and the premenstrual period, and is particularly common among multiparous women of reproductive age. Online health information on this condition is frequently accessed by both patients and healthcare professionals. This study aimed to evaluate the readability, quality, and reliability of responses generated by three popular artificial intelligence (AI) chatbots-ChatGPT-4o, Gemini, and Perplexity-regarding the most commonly asked questions on PCS.MethodsFrom 25 PCS-related keywords identified through Google Trends, duplicates and irrelevant terms were excluded, yielding a final set of 18 keywords. These were separately submitted to the three AI chatbots. Readability was assessed using the Gunning Fog Index (GFOG), Simple Measure of Gobbledygook (SMOG), and Flesch Reading Ease Score (FRES), among other indices, calculated via two independent readability tools. Quality was measured using the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) tool, while reliability was assessed with the Journal of the American Medical Association (JAMA) benchmark criteria and the modified DISCERN instrument.ResultsReadability analyses revealed that all chatbot responses significantly exceeded the sixth-grade level recommended by the AMA and NIH (p < .001). ChatGPT-4o achieved the highest readability on the CLI and ARI, whereas Gemini performed better on the remaining indices, and Perplexity consistently produced the most complex responses. Among the three evaluated freely accessible platforms, Perplexity achieved the highest scores on several quality and reliability measures (JAMA, DISCERN, and GQS), while Gemini outperformed the others on EQIP scoring.ConclusionAlthough AI chatbots hold promise for providing accessible information on PCS, their readability levels remain above recommended standards, and overall quality scores are suboptimal. Producing clearer, more balanced, and more reliable content could enhance the contribution of these technologies to public health.