AI-Driven Information for Relatives of Patients with Malignant Middle Cerebral Artery Infarction: A Preliminary Validation Study Using GPT-4o.

Journal: Brain sciences
Published Date: Apr 11, 2025

Abstract

This study examines GPT-4o's ability to communicate effectively with relatives of patients undergoing decompressive hemicraniectomy (DHC) after malignant middle cerebral artery infarction (MMCAI). GPT-4o was asked 25 common questions from patients' relatives about DHC for MMCAI, twice over a 7-day interval. Responses were rated for accuracy, clarity, relevance, completeness, sourcing, and usefulness by board-certified intensivist* (one), neurologists, and neurosurgeons using the Quality Analysis of Medical AI (QAMAI) tool. Interrater reliability and stability were measured using ICC and Pearson's correlation. The total QAMAI scores were 22.32 ± 3.08 for the intensivist, 24.68 ± 2.8 for the neurologist, 23.36 ± 2.86 and 26.32 ± 2.91 for the neurosurgeons, representing moderate-to-high accuracy. The evaluators reported moderate ICC (0.631, 95% CI: 0.321-0.821). The highest subscores were for the categories of accuracy, clarity, and relevance while the poorest were associated with completeness, usefulness, and sourcing. GPT-4o did not systematically provide references for their responses. The stability analysis reported moderate-to-high stability. The readability assessment revealed an FRE score of 7.23, an FKG score of 15.87 and a GF index of 18.15. GPT-4o provides moderate-to-high quality information related to DHC for MMCAI, with strengths in accuracy, clarity, and relevance. However, limitations in completeness, sourcing, and readability may impact its effectiveness in patient or their relatives' education.

Authors

  • Mejdeddine Al Barajraji
    Department of Neurosurgery, University Hospital of Lausanne and University of Lausanne, 1005 Lausanne, Switzerland.
  • Sami Barrit
    Department of Neurosurgery, CHU Tivoli, 7110 La Louvière, Belgium.
  • Nawfel Ben-Hamouda
    Department of Adult Intensive Care, University Hospital of Lausanne (CHUV), University of Lausanne, 1005 Lausanne, Switzerland.
  • Ethan Harel
    Department of Neurosurgery, University Hospital of Lausanne and University of Lausanne, 1005 Lausanne, Switzerland.
  • Nathan Torcida
    Department of Neurology, Hôpital Universitaire de Bruxelles (HUB), 1070 Brussels, Belgium.
  • Beatrice Pizzarotti
    Department of Neurology, University Hospital of Lausanne (CHUV), University of Lausanne, 1011 Lausanne, Switzerland.
  • Nicolas Massager
    Department of Neurosurgery, CHU Tivoli, 7110 La Louvière, Belgium.
  • Jérôme R Lechien
    Department of Otolaryngology-Head and Neck Surgery, Division of Laryngology and Bronchoesophagology, Baudour, Belgium.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals