Uncertainty quantification for detecting out-of-distribution cases in deep learning segmentation of head and neck organs of interest.

PURPOSE: Quality assurance (QA) of auto-segmentations is necessary for safe clinical deployment. Deep learning models are optimised for data similar to their training set, but reliability drops with out-of-distribution (OOD) inputs. This study evaluates whether uncertainty quantification (UQ) can detect OOD inputs arising from distribution shift in head and neck organ of interest (OOI) auto-segmentation. MATERIALS AND METHODS: Monte Carlo dropout-based uncertainty scores and geometric accuracy of nnU-Net models were evaluated in two experiments. First, as a proof-of-concept, we assessed whether uncertainty scores detect metal artefact (N = 10) from non-artefact cases (N = 10) on a baseline model (mixed training data, N = 620) and a no-metal model (artefact-free training data, N = 620). Second, in a clinically realistic experiment, we split a fine-tuning dataset by uncertainty score to obtain a narrow model (low-uncertainty cases, N = 21) and a broad model (high-uncertainty cases, N = 21). A held-out test set (N = 76) evaluated whether segmentation accuracy differed between uncertainty categories. RESULTS: Both experiments showed that uncertainty scores reflect the learned training distribution. In the supervised experiment, the no-metal model demonstrated greater uncertainty score separation between artefact and non-artefact cases (1.1 × 105 vs 0.6 × 105) than the baseline model (1.1 × 105 vs 0.8 × 105). In the unsupervised experiment, the narrow model performed significantly better on certain than uncertain cases (p = 0.04), while no difference was seen for the broad model (p = 0.34), which did include these cases in training. CONCLUSIONS: Uncertainty quantification can detect distribution shifts in radiotherapy auto-segmentation. This could support automated QA workflows by identifying in-distribution cases and enabling the application of specialised models.