Evaluation of the quality of information provided by ChatGPT on distal biceps repair surgery.

BACKGROUND: Distal biceps tendon rupture is a rare injury, typically occurring during forceful, eccentric contraction of the biceps brachii muscle. This study aimed to assess the information AI software (ChatGPT) provides when searching for distal bicep injuries and their management. Using standardised scoring systems, we evaluated the quality of the information provided, its trustworthiness, and its readability. METHODS: An open AI model (ChatGPT) was used to answer 25 commonly asked questions from patients about distal biceps surgery. These answers were evaluated for medical accuracy, quality, and readability using the JAMA Benchmark criteria, DISCERN score, Flesch-Kincaid Reading Ease Score (FRES), and Grade Level (FKGL). RESULTS: The JAMA Benchmark criteria score was 0, the lowest score, indicating no reliable resources cited. The DISCERN score was 44.3, which is considered a good score. The areas in the open AI model that did not achieve full marks were related to the lack of available source material used to compile the answers and, finally, some shortcomings with information not fully supported by the literature. The FRES was 38.5, and the FKGL was at a college reading level. CONCLUSION: A high reading level was required to comprehend the information provided by ChatGPT about distal biceps repair, and the evidence supplied was of fair quality. With no citations provided, it remains unclear where these answers originate. However, ChatGPT continued to safety-net patients by encouraging the importance of further discussion with a surgeon. More high-quality sources that patients can easily comprehend are required to educate patients concerning distal bicep repair.