Enhancing Large Language Models with Retrieval-Augmented Generation: A Radiology-Specific Approach.

Journal: Radiology. Artificial intelligence
PMID:

Abstract

Retrieval-augmented generation (RAG) is a strategy to improve the performance of large language models (LLMs) by providing an LLM with an updated corpus of knowledge that can be used for answer generation in real time. RAG may improve LLM performance and clinical applicability in radiology by providing citable, up-to-date information without requiring model fine-tuning. In this retrospective study, a radiology-specific RAG system was developed using a vector database of 3689 articles published from January 1999 to December 2023. Performance of five LLMs with (RAG-Systems) and without RAG on a 192-question radiology examination was compared. RAG significantly improved examination scores for GPT-4 (OpenAI; 81.2% vs 75.5%, = .04) and Command R+ (Cohere; 70.3% vs 62.0%, = .02), but not for Claude Opus (Anthropic), Mixtral (Mistral AI), or Gemini 1.5 Pro (Google DeepMind). RAG-Systems performed significantly better than pure LLMs on a 24-question subset directly sourced from (85% vs 76%, = .03). RAG-Systems retrieved 21 of 24 (87.5%, < .001) relevant references cited in the examination's answer explanations and successfully cited them in 18 of 21 (85.7%, < .001) outputs. The results suggest that RAG is a promising approach to enhance LLM capabilities for radiology knowledge tasks, providing transparent, domain-specific information retrieval. Computer Applications-General (Informatics), Technology Assessment © RSNA, 2025 See also commentary by Mansuri and Gichoya in this issue.

Authors

  • Dane A Weinert
    Department of Radiology, University of Southern California, Keck School of Medicine, 1510 San Pablo St, Ste 350, Los Angeles, CA 90033.
  • Andreas M Rauschecker
    Department of Radiology, Hospital of the University of Pennsylvania, 3400 Spruce St, Philadelphia, PA 19104 (J.D.R., L.X., A.K., J.M.E., T.C., I.M.N., S.M., J.C.G.); Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, Calif (J.D.R., A.M.R.); Penn Image Computing and Science Laboratory, University of Pennsylvania, Philadelphia, Pa (X.L., J.W.); University of Pennsylvania Perelman School of Medicine, Philadelphia, Pa (M.T.D.); Mecklenburg Radiology Associates, Charlotte, NC (E.J.B.); Department of Radiology, University of Texas, Austin, Tex (R.N.B.); and Division of Nuclear Medicine and Clinical Molecular Imaging, Department of Radiology, University of Pennsylvania, Philadelphia, Pa (I.M.N.).