Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions
Journal:
arXiv
Published Date:
Feb 1, 2025
Abstract
Emerging research has highlighted that artificial intelligence based
multimodal fusion of digital pathology and transcriptomic features can improve
cancer diagnosis (grading/subtyping) and prognosis (survival risk) prediction.
However, such direct fusion for joint decision is impractical in real clinical
settings, where histopathology is still the gold standard for diagnosis and
transcriptomic tests are rarely requested, at least in the public healthcare
system. With our novel diffusion based crossmodal generative AI model PathGen,
we show that genomic expressions synthesized from digital histopathology
jointly predicts cancer grading and patient survival risk with high accuracy
(state-of-the-art performance), certainty (through conformal coverage
guarantee) and interpretability (through distributed attention maps). PathGen
code is available for open use by the research community through GitHub at
https://github.com/Samiran-Dey/PathGen.
