Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer
Journal:
arXiv
Published Date:
May 2, 2025
Abstract
This study proposes a novel approach combining Multimodal Deep Learning with
intrinsic eXplainable Artificial Intelligence techniques to predict
pathological response in non-small cell lung cancer patients undergoing
neoadjuvant therapy. Due to the limitations of existing radiomics and unimodal
deep learning approaches, we introduce an intermediate fusion strategy that
integrates imaging and clinical data, enabling efficient interaction between
data modalities. The proposed Multimodal Doctor-in-the-Loop method further
enhances clinical relevance by embedding clinicians' domain knowledge directly
into the training process, guiding the model's focus gradually from broader
lung regions to specific lesions. Results demonstrate improved predictive
accuracy and explainability, providing insights into optimal data integration
strategies for clinical applications.
