Screening/diagnosis of pediatric endocrine disorders through the artificial intelligence model in different language settings.

Journal: European journal of pediatrics
Published Date: Mar 19, 2024

Abstract

UNLABELLED: This study is aimed at examining the impact of ChatGPT on pediatric endocrine and metabolic conditions, particularly in the areas of screening and diagnosis, in both Chinese and English modes. A 40-question questionnaire covering the four most common pediatric endocrine and metabolic conditions was posed to ChatGPT in both Chinese and English three times each. Six pediatric endocrinologists evaluated the responses. ChatGPT performed better when responding to questions in English, with an unreliable rate of 7.5% compared to 27.5% for Chinese questions, indicating a more consistent response pattern in English. Among the reliable questions, the answers were more comprehensive and satisfactory in the English mode. We also found disparities in ChatGPT's performance when interacting with different target groups and diseases, with improved performance for questions posed by clinicians in English and better performance for questions related to diabetes and overweight/obesity in Chinese for both clinicians and patients. Language comprehension, providing incomprehensive answers, and errors in key data were the main contributors to the low scores, according to reviewer feedback.

Authors

  • Lingwen Ying
    Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China.
  • Sichen Li
    Department of Neurosurgery, Huashan Hospital, Shanghai Medical College, Fudan University, Shanghai, 200040, China.
  • Chunyang Chen
    Faculty of Information Technology, Monash University, Clayton, VIC, 3800, Australia.
  • Fan Yang
    School of Electrical Engineering and Automation, Jiangsu Normal University, Xuzhou, China.
  • Xin Li
    Veterinary Diagnostic Center, Shanghai Animal Disease Control Center, Shanghai, China.
  • Yao Chen
    Department of Galactophore Surgery, West China Hospital, Sichuan University, Chengdu, 610041, PR China.
  • Yu Ding
    College of Economics and Management, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China.
  • Guoying Chang
    Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China. wangxiumin1019@126.com.
  • Juan Li
    Department of Hygienic Inspection, School of Public Health, Jilin University 1163 Xinmin Street Changchun 130021 Jilin China songxiuling@jlu.edu.cn li_juan@jlu.edu.cn jinmh@jlu.edu.cn +86 43185619441.
  • Xiumin Wang
    Department of Endocrinology and Metabolism, Shanghai Children's Medical Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200127, China. changguoying@126.com.

Keywords

External Resources

Popular Topics
Recent Journals