Robustness of a neural network approved for skin cancer diagnosis against variations in sequential images.

BACKGROUND: Convolutional neural networks (CNN) for skin cancer classification have shown results comparable to dermatologists but are vulnerable to minor image transformations. We investigated the robustness of a MDR class-IIa certified CNN when classifying sequential images of identical lesions. PATIENTS AND METHODS: We acquired 2,744 dermoscopic images of 385 skin lesions (80.8 % benign, 19.2 % malignant) and applied in-vivo zoom, rotation (90-degree increments), and simple repetitions of image recordings. Sequential images of identical lesions were classified by a binary CNN (Moleanalyzer-Pro, FotoFinder Systems, Germany) and the variability of scores was investigated using intraclass correlation coefficient (ICC), mean absolute change of scores (mac), and probability of change of predicted class ( π c l a s s c h a n g e ${{\pi }_{class\ change}}$ ). RESULTS: In dermoscopic baseline images (n = 385) the CNN showed a sensitivity, specificity, and area under the receiver operating characteristic (AUROC) (95 % CI) of 91.9 % (83.4 %-96.2 %), 87.8 % (83.7 %-91.0 %) and 0.947 (0.921-0.972), respectively. The ICC across images of identical lesions was 0.872 (0.862-0.883), indicating excellent reliability. Overall mac of scores was 0.102 (0.090-0.115) and π c l a s s c h a n g e ${{\pi }_{class\ change}}$ was 7.5 % (5.8 %-9.2 %). CONCLUSIONS: The tested CNN demonstrated a profound robustness against image variations as might be introduced during sequential digital dermoscopy. Clinically relevant class changes occurred in one of 13 images.