A multi-centre real-world evaluation of AI-assisted organ at risk contouring on radiotherapy treatment planning workflows.

OBJECTIVES: This multicentre real-world evaluation, commissioned by NHS England, evaluated the impact of AI-assisted contouring of organs at risk on contour acceptability, workflows, and staffing. METHODS: Data from 626 patients were collected from eight NHS radiotherapy departments. Time metrics from the date of planning CT scan to the start of first treatment were compared between manual and AI-enabled pathways. Acceptability scores for AI-generated contours were also collected. RESULTS: AI-assisted contouring increased potential efficiency in treatment planning compared to manual methods, reducing time for contouring and redistributing workload across different staff types. 100% of manual contour reviews involved clinical oncologists compared to 38% reviewing AI-generated contours. However, workflow design meant that time saving was not observed across the whole treatment planning pathway. AI contours were generally well accepted, with 15.9% requiring no edits and 64.4% only minor edits. This varied by anatomy, with breast having the best acceptability and prostate and head and neck contours requiring more editing. CONCLUSIONS: AI contouring tools have the potential to enhance efficiency in radiotherapy treatment planning, creating operational flexibility. Pathway review and revision could unlock further benefits, for example, involving different staff types to address local bottlenecks. Workflow, capacity, and staffing review pre- and post-implementation could increase efficiency gains with AI-assisted contouring. ADVANCES IN KNOWLEDGE: This study evaluated AI contouring tools in complex, real-world systems which differ between departments for generalizable conclusions. It describes the changes in time across the whole treatment planning pathway and the system-level impact of using AI tools to help inform holistic planning.