Real-world clinical impact of implementing and updating a deep learning-based automatic contouring system in rectal cancer radiotherapy.

BACKGROUND: Accurate delineation of target volumes and organs-at-risk (OARs) is a critical yet labor-intensive component of rectal cancer radiotherapy. While deep learning (DL)-based automatic contouring systems are increasingly used to address inter-observer variability and improve efficiency, artificial intelligence models require rigorous quality assurance and updates to reflect current technology. However, high-level evidence regarding the longitudinal real-world impact of implementing and iteratively updating these systems in clinical workflows is currently lacking. PURPOSE: This study aimed to evaluate the real-world clinical impact of implementing and updating a DL-based automatic contouring system in rectal cancer radiotherapy to generate high-quality evidence of iterative updates. METHODS: This longitudinal retrospective analysis included 150 patients divided into three cohorts: pre-implementation (n1 = 50), post-implementation (n2 = 50), and post-update (n3 = 50). Geometric similarities between unedited-automatic and final treatment contours were compared across cohorts. Failure rates were systematically analyzed. Six oncologists contoured 21 additional cases through manual, first-generation (Auto1), and second-generation (Auto2) system-assisted methods to evaluate contouring time, inter-observer consistency, and accuracy. Additionally, a 5-point Likert scale was used by two blinded senior oncologists to assess the clinical acceptability of the generated contours. RESULTS: The mean Dice similarity coefficient (DSC) values of clinical target volume (CTV) before and after implementing the automatic contouring system were 0.87 ± 0.04 and 0.88 ± 0.04 (P = 0.067), while those of OARs were 0.80 ± 0.06 and 0.88 ± 0.05 (P < 0.001), respectively. Following the system update, they improved from 0.88 ± 0.04 to 0.93 ± 0.04 for CTV (P < 0.001) and from 0.88 ± 0.05 to 0.95 ± 0.02 for OARs (P < 0.001). The system update achieved an approximately 80.6% reduction in the mean failure rate. Auto2-assisted method decreased the total time by approximately 58.8% compared with the manual method, and 21.9% compared with the Auto1-assisted method. This method also demonstrated optimal inter-observer consistency (0.95 ± 0.03) and accuracy (0.94 ± 0.03) for CTV. In the blinded clinical evaluation, 99.2% (125/126) of the oncologist-revised final contours received a Likert score of ≥ 4, and Auto2-generated unedited contours showed significantly higher clinical acceptability than Auto1 (4.02 ± 0.25 vs. 3.26 ± 0.49, P < 0.001) CONCLUSIONS: Implementing an automatic contouring system provided crucial guidance for clinical practice. Its iterative update significantly reduced workload and inter-observer variation while enhancing contouring efficiency and quality.