A Language Vision Model Approach for Automated Tumor Contouring in Radiation Oncology
Journal:
arXiv
Published Date:
Mar 19, 2025
Abstract
Background: Lung cancer ranks as the leading cause of cancer-related
mortality worldwide. The complexity of tumor delineation, crucial for radiation
therapy, requires expertise often unavailable in resource-limited settings.
Artificial Intelligence(AI), particularly with advancements in deep learning
(DL) and natural language processing (NLP), offers potential solutions yet is
challenged by high false positive rates. Purpose: The Oncology Contouring
Copilot (OCC) system is developed to leverage oncologist expertise for precise
tumor contouring using textual descriptions, aiming to increase the efficiency
of oncological workflows by combining the strengths of AI with human oversight.
Methods: Our OCC system initially identifies nodule candidates from CT scans.
Employing Language Vision Models (LVMs) like GPT-4V, OCC then effectively
reduces false positives with clinical descriptive texts, merging textual and
visual data to automate tumor delineation, designed to elevate the quality of
oncology care by incorporating knowledge from experienced domain experts.
Results: Deployments of the OCC system resulted in a significant reduction in
the false discovery rate by 35.0%, a 72.4% decrease in false positives per
scan, and an F1-score of 0.652 across our dataset for unbiased evaluation.
Conclusions: OCC represents a significant advance in oncology care,
particularly through the use of the latest LVMs to improve contouring results
by (1) streamlining oncology treatment workflows by optimizing tumor
delineation, reducing manual processes; (2) offering a scalable and intuitive
framework to reduce false positives in radiotherapy planning using LVMs; (3)
introducing novel medical language vision prompt techniques to minimize LVMs
hallucinations with ablation study, and (4) conducting a comparative analysis
of LVMs, highlighting their potential in addressing medical language vision
challenges.
