Evaluating the Accuracy and Readability of ChatGPT in Addressing Patient Queries on Adult Spinal Deformity Surgery.

Journal: Global spine journal
Published Date: Jul 11, 2025

Abstract

Study DesignCross-Sectional.ObjectivesAdult spinal deformity (ASD) affects 68% of the elderly, with surgical intervention carrying complication rates of up to 50%. Effective patient education is essential for managing expectations, yet high patient volumes can limit preoperative counseling. Language learning models (LLMs), such as ChatGPT, may supplement patient education. This study evaluates ChatGPT-3.5's accuracy and readability in answering common patient questions regarding ASD surgery.MethodsStructured interviews with ASD surgery patients identified 40 common preoperative questions, of which 19 were selected. Each question was posed to ChatGPT-3.5 in separate chat sessions to ensure independent responses. Three spine surgeons assessed response accuracy using a validated 4-point scale (1 = excellent, 4 = unsatisfactory). Readability was analyzed using the Flesch-Kincaid Grade Level formula.ResultsPatient inquiries fell into four themes: (1) Preoperative preparation, (2) Recovery (pain expectations, physical therapy), (3) Lifestyle modifications, and (4) Postoperative course. Accuracy scores varies: Preoperative responses averaged 1.67, Recovery and lifestyle responses 1.33, and postoperative responses 2.0. 59.7% of responses were excellent (no clarification needed), 26.3% were satisfactory (minimal clarification needed), 12.3% required moderate clarification, and 1.8% were unsatisfactory, with one response ("Will my pain return or worsen?") rated inaccurate by all reviewers. Readability analysis showed all 19 responses exceeded the eight-grade reading level by an average of 5.91 grade levels.ConclusionChatGPT-3.5 demonstrates potential as a supplemental patient education tool but provides varying accuracy and complex readability. While it may support patient understanding, the complexity of its responses may limit usefulness for individuals with lower health literacy.

Authors

  • Fergui Hernandez
    Department of Orthopaedic Surgery, Keck School of Medicine of the University of Southern California, Los Angeles, CA, USA.
  • Rafael Guizar
    Department of Orthopaedic Surgery, Keck School of Medicine of the University of Southern California, Los Angeles, CA, USA.
  • Henry Avetisian
    Department of Orthopaedic Surgery, Keck School of Medicine at the University of Southern California, Los Angeles, CA, USA.
  • Marc A Abdou
    Department of Orthopaedic Surgery, Keck School of Medicine of the University of Southern California, Los Angeles, CA, USA.
  • William J Karakash
    Department of Orthopaedic Surgery, Keck School of Medicine at the University of Southern California, Los Angeles, CA, USA.
  • Andy Ton
    Department of Orthopaedic Surgery, University of California, Irvine, CA, USA.
  • Matthew C Gallo
    Department of Orthopaedic Surgery, Keck School of Medicine of the University of Southern California, Los Angeles, CA, USA.
  • Jacob R Ball
    Department of Orthopaedic Surgery, Keck School of Medicine of the University of Southern California, Los Angeles, CA, USA.
  • Jeffrey C Wang
    Department of Orthopaedic Surgery, University of Southern California, Orthopaedic SurgeryHC4 -1450 San Pablo St, #5400 Health Sciences Campus, Los Angeles, CA 90033, USA.
  • Ram K Alluri
    Department of Orthopaedic Surgery, Keck School of Medicine at the University of Southern California, Los Angeles, CA, USA.
  • Raymond J Hah
    Department of Orthopaedic Surgery, Keck School of Medicine at the University of Southern California, Los Angeles, CA, USA.
  • Michael Safaee
    Department of Neurosurgery, Keck School of Medicine of the University of Southern California, Los Angeles, CA, USA.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals