Global explainability of a deep abstaining classifier
Journal:
arXiv
Published Date:
Apr 1, 2025
Abstract
We present a global explainability method to characterize sources of errors
in the histology prediction task of our real-world multitask convolutional
neural network (MTCNN)-based deep abstaining classifier (DAC), for automated
annotation of cancer pathology reports from NCI-SEER registries. Our classifier
was trained and evaluated on 1.04 million hand-annotated samples and makes
simultaneous predictions of cancer site, subsite, histology, laterality, and
behavior for each report. The DAC framework enables the model to abstain on
ambiguous reports and/or confusing classes to achieve a target accuracy on the
retained (non-abstained) samples, but at the cost of decreased coverage.
Requiring 97% accuracy on the histology task caused our model to retain only
22% of all samples, mostly the less ambiguous and common classes. Local
explainability with the GradInp technique provided a computationally efficient
way of obtaining contextual reasoning for thousands of individual predictions.
Our method, involving dimensionality reduction of approximately 13000
aggregated local explanations, enabled global identification of sources of
errors as hierarchical complexity among classes, label noise, insufficient
information, and conflicting evidence. This suggests several strategies such as
exclusion criteria, focused annotation, and reduced penalties for errors
involving hierarchically related classes to iteratively improve our DAC in this
complex real-world implementation.
