Identifying Unmonitored Tone Drift in Patient Education Materials Transcribed to a Lower Reading Level by a Large Language Model.

BACKGROUND AND OBJECTIVES: Effective patient education is essential in neurosurgery, but many materials exceed recommended readability levels, which can limit comprehension and informed consent. Simplification can also alter tone, potentially introducing bias. Recent studies have used large language models such as Chat Generative Pre-trained Transformer (ChatGPT) to simplify neurosurgical patient education materials (PEMs), but the impact of this process on sentiment and emotional tone remains unclear. Our objective was to assess the sentiment and emotional tone of neurosurgical PEMs before and after conversion to a lower reading level by ChatGPT. METHODS: A total of 336 neurosurgical PEMs covering stroke, spinal stenosis, hydrocephalus, epilepsy, and pituitary brain tumors were analyzed for readability, sentiment, and emotion. Each was then simplified to a seventh grade level using GPT-4.0. Readability was evaluated using Flesch-Kincaid Grade, Flesch Reading Ease, Gunning Fog Index, Automated Readability Index, Coleman-Liau Index, and Simple Measure of Gobbledygook. Sentiment and emotional tone were described using the Valence Aware Dictionary and sEntiment Reasoner (VADER) algorithm and National Research Council Canada Emotion Lexicon. Paired statistical t-tests assessed the significance of changes. RESULTS: Simplification produced substantial improvements in readability across all 6 indices and all neurosurgical topics (P < .001). Sentiment shifted toward increased positivity, reflected by higher VADER compound scores, more positive tokens, and fewer neutral tokens. Disgust decreased significantly across every topic, whereas sadness, surprise, and joy increased modestly; fear and anger showed no significant change. Topic-level analyses mirrored global patterns, demonstrating consistent directional effects. Overall, simplification achieved large readability gains while introducing small but measurable alterations in emotional tone. CONCLUSION: The decrease in neutral and negative sentiment suggests a shift toward more persuasive language. Modest but consistent shifts in sentiment and emotional tone accompanying artificial intelligence-assisted simplification highlight the potential for unintended affective shifts during artificial intelligence simplification and warrant monitoring when deploying large language models for patient-facing materials. Current PEMs pose a communication barrier between patient and provider, but providers must be careful.