Retrieval-augmented generation enhances large language model performance on the Japanese orthopedic board examination.

Journal: Journal of orthopaedic science : official journal of the Japanese Orthopaedic Association
Published Date: Mar 28, 2025

Abstract

INTRODUCTION: Large language models (LLMs) have shown potential in medical applications. However, their effectiveness in specialized medical domains remains underexplored. The integration of Retrieval-Augmented Generation (RAG) has been proposed to improve these models by reducing hallucinations and enhancing domain-specific information access. Through this evaluation, we aim to assess whether RAG can effectively bridge the gap between LLMs' current capabilities and the accuracy needed for medical use by examining GPT-3.5 Turbo, GPT-4o, and o1-preview on the 2024 Japanese Orthopedic Specialist Examination.

Authors

  • Juntaro Maruyama
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Satoshi Maki
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan; Center for Frontier Medical Engineering, Chiba University, Japan. Electronic address: satoshimaki@gmail.com.
  • Takeo Furuya
    Department of Orthopaedic Surgery, Graduate School of Medicine, Chiba, Japan.
  • Yuki Nagashima
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Kyota Kitagawa
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Yasunori Toki
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Shuhei Iwata
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Megumi Yazaki
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Takaki Kitamura
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Sho Gushiken
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Yuji Noguchi
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Masataka Miura
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Masahiro Inoue
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Yasuhiro Shiga
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Kazuhide Inage
    Department of Orthopedic Surgery, Graduate School of Medicine, Chiba University, Japan.
  • Sumihisa Orita
    Department of Orthopaedic Surgery, Graduate School of Medicine, Chiba, Japan.
  • Seiji Ohtori
    Department of Orthopaedic Surgery, Graduate School of Medicine, Chiba, Japan.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals