Step into the era of large multimodal models: a pilot study on ChatGPT-4V(ision)'s ability to interpret radiological images.

Journal: International journal of surgery (London, England)
Published Date: Jul 1, 2024

Abstract

BACKGROUND: The introduction of ChatGPT-4V's 'Chat with images' feature represents the beginning of the era of large multimodal models (LMMs), which allows ChatGPT to process and answer questions based on uploaded images. This advancement has the potential to transform how surgical teams utilize radiographic data, as radiological interpretation is crucial for surgical planning and postoperative care. However, a comprehensive evaluation of ChatGPT-4V's capabilities in interpret radiological images and formulating treatment plans remains to be explored.

Authors

  • Lingxuan Zhu
    Department of Urology, Renji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200127, China; Department of Etiology and Carcinogenesis, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China; Changping Laboratory, Beijing, China.
  • Weiming Mou
    Department of Urology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
  • Yancheng Lai
    Department of Urology, Renji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200127, China; The First School of Clinical Medicine, Southern Medical University, Guangzhou, China.
  • Jinghong Chen
    Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou.
  • Shujia Lin
    Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou.
  • Liling Xu
    Tongren Hospital Shanghai Jiao Tong University, Shanghai, China.
  • Junda Lin
    Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou.
  • Zeji Guo
    Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou.
  • Tao Yang
    The First Clinical Medical College, The Affiliated People's Hospital of Fujian University of Traditional Chinese Medicine, Fuzhou, Fujian, China.
  • Anqi Lin
    Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou, China.
  • Chang Qi
    Institute of Logic and Computation, TU Wien, Austria.
  • Ling Gan
    Department of Ultrasound Medicine, The First Affiliated Hospital, Fujian Medical University, Fujian.
  • Jian Zhang
    College of Pharmacy, Ningxia Medical University, Yinchuan, NingxiaHui Autonomous Region, China.
  • Peng Luo
    Institute of Forest Resource Information Techniques, Chinese Academy of Forestry, Beijing 100091, PR China.

Keywords

External Resources

Popular Topics
Recent Journals