Integration of deep learning and radiomic features from multiplex immunohistochemistry images for reproducible Multi-Outcome prediction in a Multi-Center study of colorectal cancer.

OBJECTIVE: To develop and validate a robust, multimodal machine learning framework integrating radiomic and deep learning features from multiplex immunohistochemistry (mIHC) images for comprehensive outcome prediction in colorectal cancer (CRC). MATERIALS AND METHODS: This multi-institutional retrospective study included 2,117 CRC patients from seven centers, with 1,548 cases used for model training and internal testing, and 569 for external validation. mIHC-stained whole-slide images targeting six immune markers (CD3, CD8, CD45RO, PD-1, LAG-3, Tim-3) were analyzed from two spatial compartments: tumor center and invasive margin. Radiomic features (n = 71/region/marker) were extracted using HistomicsTK, while 768-dimensional deep features were derived using a pre-trained Vision Transformer (ViT-B/16). Feature robustness across biomarkers was quantified via intraclass correlation coefficients (ICC ≥ 0.75). Selected features underwent multi-step selection (LASSO, MI, RFE) and were fused into a single feature space, followed by PCA-based dimensionality reduction. Five clinical tasks were modeled: tumor recurrence, survival status, overall survival duration, TNM staging, and immune profile classification. Classification models (TabTransformer, XGBoost, TabNet) and survival models (DeepSurv, CoxPH, RSF) were trained using 5-fold cross-validation and tested on independent cohorts. RESULTS: Fused features significantly outperformed individual modalities across all tasks. TabTransformer with LASSO-selected fused features achieved top performance: recurrence (AUC = 95.9%), survival status (AUC = 94.5%), TNM staging (macro-AUC = 91.0%), and immune profile (macro-AUC = 91.0%). For survival regression, DeepSurv achieved a C-index of 0.82 and time-dependent AUC of 0.82. Models exhibited strong generalizability, with negligible performance drop on external datasets. SHAP analysis confirmed feature interpretability, with fused features contributing the most across tasks. CONCLUSIONS: This study demonstrates that fused mIHC-derived radiomic and deep features yield accurate, interpretable, and generalizable predictions for multiple CRC outcomes, supporting their integration into precision oncology workflows.