Artificially Generated Visual Scanpath Improves Multi-label Thoracic Disease Classification in Chest X-Ray Images
Journal:
arXiv
Published Date:
Mar 1, 2025
Abstract
Expert radiologists visually scan Chest X-Ray (CXR) images, sequentially
fixating on anatomical structures to perform disease diagnosis. An automatic
multi-label classifier of diseases in CXR images can benefit by incorporating
aspects of the radiologists' approach. Recorded visual scanpaths of
radiologists on CXR images can be used for the said purpose. But, such
scanpaths are not available for most CXR images, which creates a gap even for
modern deep learning based classifiers. This paper proposes to mitigate this
gap by generating effective artificial visual scanpaths using a visual scanpath
prediction model for CXR images. Further, a multi-class multi-label classifier
framework is proposed that uses a generated scanpath and visual image features
to classify diseases in CXR images. While the scanpath predictor is based on a
recurrent neural network, the multi-label classifier involves a novel iterative
sequential model with an attention module. We show that our scanpath predictor
generates human-like visual scanpaths. We also demonstrate that the use of
artificial visual scanpaths improves multi-class multi-label disease
classification results on CXR images. The above observations are made from
experiments involving around 0.2 million CXR images from 2 widely-used datasets
considering the multi-label classification of 14 pathological findings. Code
link: https://github.com/ashishverma03/SDC
