Assessing the readability of AI-generated medication counseling for common Medicaid drug classes: metformin, lisinopril, and atorvastatin.

BACKGROUND: Metformin, lisinopril, and atorvastatin rank among the most commonly prescribed medications for Medicaid patients; however, patients often have inadequate knowledge regarding their safe usage. Generative artificial intelligence (AI) tools are becoming a prevalent source of medication information, but their readability for low-literacy populations compared to authoritative drug information sources lacks empirical assessment. METHODS: Sixteen standardized medication counseling inquiries (5 for metformin, 5 for lisinopril, and 6 for atorvastatin) were presented to three AI platforms (ChatGPT, Google Gemini, and Meta Llama 4) under default and sixth-grade-prompted conditions, as well as to six pharmaceutical information websites (FDA.gov, MedlinePlus, Drugs.com, Mayo Clinic, WebMD, and RxList). The Flesch-Kincaid Grade Level (FKGL) was calculated automatically from extracted passages utilizing Python textstat v0.7.3. Identical normalized-text duplicates within each source folder were omitted (84 exclusions; final unique source count = 12). The Mann-Whitney U and Wilcoxon signed-rank tests evaluated FKGL disparities (α = 0.05). RESULTS: The average pooled default AI FKGL was 12.19 ± 4.85, while the pooled sixth-grade AI FKGL was 7.63 ± 3.16. The pooled website FKGL after deduplication was 10.49 ± 3.95 (Mann-Whitney U = 343.0, P = 0.31, rank-biserial r = -0.19). CONCLUSION: The AI-generated medication counseling passages in this sample frequently exhibited a higher FKGL than the aggregated website passages; sixth-grade prompting markedly decreased FKGL across all three models (all P ≤ 0.001). This article exclusively addresses readability. Pharmacists and prescribers should endorse reputable websites that provide suitable assistance; grade-level prompts may facilitate patient comprehension of AI-generated summaries.