Cross-Modality Masked Learning for Survival Prediction in ICI Treated NSCLC Patients
Journal:
arXiv
Published Date:
Jul 9, 2025
Abstract
Accurate prognosis of non-small cell lung cancer (NSCLC) patients undergoing
immunotherapy is essential for personalized treatment planning, enabling
informed patient decisions, and improving both treatment outcomes and quality
of life. However, the lack of large, relevant datasets and effective
multi-modal feature fusion strategies pose significant challenges in this
domain. To address these challenges, we present a large-scale dataset and
introduce a novel framework for multi-modal feature fusion aimed at enhancing
the accuracy of survival prediction. The dataset comprises 3D CT images and
corresponding clinical records from NSCLC patients treated with immune
checkpoint inhibitors (ICI), along with progression-free survival (PFS) and
overall survival (OS) data. We further propose a cross-modality masked learning
approach for medical feature fusion, consisting of two distinct branches, each
tailored to its respective modality: a Slice-Depth Transformer for extracting
3D features from CT images and a graph-based Transformer for learning node
features and relationships among clinical variables in tabular data. The fusion
process is guided by a masked modality learning strategy, wherein the model
utilizes the intact modality to reconstruct missing components. This mechanism
improves the integration of modality-specific features, fostering more
effective inter-modality relationships and feature interactions. Our approach
demonstrates superior performance in multi-modal integration for NSCLC survival
prediction, surpassing existing methods and setting a new benchmark for
prognostic models in this context.
