Automated radiotherapy treatment planning guided by GPT-4Vision.

Journal: Physics in medicine and biology
Published Date: Jul 15, 2025

Abstract

Radiotherapy treatment planning is a time-consuming and potentially subjective process that requires the iterative adjustment of model parameters to balance multiple conflicting objectives. Recent advancements in frontier Artificial Intelligence (AI) models offer promising avenues for addressing the challenges in planning and clinical decision-making. This study introduces GPT-RadPlan, an automated treatment planning framework that integrates radiation oncology knowledge with the reasoning capabilities of large multi-modal models, such as GPT-4Vision (GPT-4V) from OpenAI. Approach: Via in-context learning, we incorporate clinical requirements and a few (3 in our experiments) approved clinical plans with their optimization settings, enabling GPT-4V to acquire treatment planning domain knowledge. The resulting GPT-RadPlan system is integrated into our in-house inverse treatment planning system through an application programming interface (API). For a given patient, GPT-RadPlan acts as both plan evaluator and planner, first assessing dose distributions and dose-volume histograms (DVHs), and then providing ``textual feedback'' on how to improve the plan to match the physician's requirements. In this manner, GPT-RadPlan iteratively refines the plan by adjusting planning parameters, such as weights and dose objectives, based on its suggestions. Main results: The efficacy of the automated planning system is showcased across 17 prostate cancer and 13 head & neck cancer VMAT plans with prescribed doses of 70.2 Gy and 72 Gy, respectively, where we compared GPT-RadPlan results to clinical plans produced by human experts. In all cases, GPT-RadPlan either outperformed or matched the clinical plans, demonstrating superior target coverage and reducing organ-at-risk doses by 5 Gy on average. Significance: Consistently satisfying the dose-volume objectives in the clinical protocol, GPT-RadPlan represents the first multimodal large language model agent that mimics the behaviors of human planners in radiation oncology clinics, achieving promising results in automating the treatment planning process without the need for additional training.

Authors

  • Sheng Liu
    Medical School, Xizang Minzu University, Xianyang, People's Republic of China.
  • Oscar Pastor-Serrano
    Delft University of Technology, Department of Radiation Science and Technology, Mekelweg 15, Delft 2629JB, Netherlands. Electronic address: o.pastorserrano@tudelft.nl.
  • Yizheng Chen
    Department of Radiation Oncology, Stanford University, Stanford, 94305, USA.
  • Matthew Gopaulchan
    Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, USA.
  • Weixin Liang
    Guangdong Provincial Key Laboratory of Chemical Measurement and Emergency Test Technology, Institute of Analysis, Guangdong Academy of Sciences (China National Analytical Center, Guangzhou), Guangzhou, 510070, China; Guangdong Provincial Engineering Research Center for Online Monitoring of Water Pollution, Institute of Analysis, Guangdong Academy of Sciences (China National Analytical Center, Guangzhou), Guangzhou, 510070, China.
  • Mark Buyyounouski
    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, People's Republic of China.
  • Erqi Pollom
    Department of Radiation Oncology, Stanford University, Stanford, CA, USA.
  • Quynh-Thu Le
    Department of Radiation Oncology, Stanford University School of Medicine , Stanford, California 94305, United States.
  • Michael Francis Gensheimer
  • Peng Dong
    Department of Radiation Oncology, Stanford University, CA, USA.
  • Yong Yang
    Department of Radiation Oncology, Stanford University, CA, USA.
  • James Zou
    Department of Biomedical Data Science, Stanford University, Stanford, California.
  • Lei Xing
    Department of Radiation Oncology, Stanford University, CA, USA.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals