Multidimensional evaluation of AI for integrated panvascular disease management: benchmarking against consensus guidelines on lipid, metabolic and antithrombotic therapy (2024-2025).

ABSTRACT: OBJECTIVES: To evaluate the performance of large language models (LLMs) in the comprehensive management (CM) of panvascular disease (PVD) across dimensions of language complexity, comprehensibility, actionability and information reliability. DESIGN: A cross-sectional benchmark evaluation study using a standardised clinical question bank derived from Chinese expert consensus statements on PVD management. SETTING: Four mainstream LLMs (DeepSeek-V.3.2, Doubao-1.8, Kimi-K2 and Qwen3-Max) were evaluated using 50 standardised clinical questions covering dyslipidaemia management, metabolic regulation, antithrombotic therapy and CM. The evaluation was conducted using the publicly available versions of the models in January 2026. PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcomes were differences among LLMs in nine validated readability metrics, including the Automated Readability Index, Flesch Reading Ease Score, Gunning Fog Index, Flesch-Kincaid Grade Level, Coleman-Liau Index, SMOG Index, Linsear Write, Dale-Chall Readability Score and Spache Readability Formula. Secondary outcomes included understandability, actionability and overall scores assessed using the Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P), information quality assessed using the DISCERN instrument, and correlations between readability and core evaluation metrics. RESULTS: A total of 200 artificial intelligence (AI)-generated responses were included in the analysis. Significant differences were observed among the AI models across all four evaluation metrics (PEMAT-Understandability Score (PEMAT-U), PEMAT-Actionability Score, PEMAT-Total Score (PEMAT-T) and Mean DISCERN Score (M Dis Score); all p<0.001). Qwen3-Max achieved the highest median scores for PEMAT-U and PEMAT-T, whereas DeepSeek-V.3.2 achieved the highest median M Dis Score. Content category had a limited influence on evaluation outcomes, with an overall difference initially observed only for PEMAT-U; however, this difference was no longer significant after Benjamini-Hochberg false discovery rate correction (FDR-adjusted p=0.136). Stratified analyses showed that intermodel differences were generally consistent across content categories, although the difference in PEMAT-U for the CM and cross-cutting issues category was no longer significant after FDR correction (FDR-adjusted p=0.052). Overall, model type was associated with substantial differences in PEMAT-P and DISCERN scores, whereas content category exerted a comparatively limited effect. CONCLUSIONS: Mainstream LLMs generate structured management recommendations with varying characteristics across readability, understandability, actionability and information quality. As this study focused on structural and linguistic attributes rather than clinical accuracy, the conclusions are limited to instrument-based assessments. Future research should validate real-world clinical utility and patient comprehension to ensure safe AI integration.