Focus on Focus: Focus-oriented Representation Learning and Multi-view Cross-modal Alignment for Glioma Grading
Journal:
arXiv
Published Date:
Aug 16, 2024
Abstract
Recently, multimodal deep learning, which integrates histopathology slides
and molecular biomarkers, has achieved a promising performance in glioma
grading. Despite great progress, due to the intra-modality complexity and
inter-modality heterogeneity, existing studies suffer from inadequate
histopathology representation learning and inefficient molecular-pathology
knowledge alignment. These two issues hinder existing methods to precisely
interpret diagnostic molecular-pathology features, thereby limiting their
grading performance. Moreover, the real-world applicability of existing
multimodal approaches is significantly restricted as molecular biomarkers are
not always available during clinical deployment. To address these problems, we
introduce a novel Focus on Focus (FoF) framework with paired pathology-genomic
training and applicable pathology-only inference, enhancing molecular-pathology
representation effectively. Specifically, we propose a Focus-oriented
Representation Learning (FRL) module to encourage the model to identify regions
positively or negatively related to glioma grading and guide it to focus on the
diagnostic areas with a consistency constraint. To effectively link the
molecular biomarkers to morphological features, we propose a Multi-view
Cross-modal Alignment (MCA) module that projects histopathology representations
into molecular subspaces, aligning morphological features with corresponding
molecular biomarker status by supervised contrastive learning. Experiments on
the TCGA GBM-LGG dataset demonstrate that our FoF framework significantly
improves the glioma grading. Remarkably, our FoF achieves superior performance
using only histopathology slides compared to existing multimodal methods. The
source code is available at https://github.com/peterlipan/FoF.
