Self-interactive learning: Fusion and evolution of multi-scale histomorphology features for molecular traits prediction in computational pathology.

Journal: Medical image analysis
Published Date: Jan 3, 2025

Abstract

Predicting disease-related molecular traits from histomorphology brings great opportunities for precision medicine. Despite the rich information present in histopathological images, extracting fine-grained molecular features from standard whole slide images (WSI) is non-trivial. The task is further complicated by the lack of annotations for subtyping and contextual histomorphological features that might span multiple scales. This work proposes a novel multiple-instance learning (MIL) framework capable of WSI-based cancer morpho-molecular subtyping by fusion of different-scale features. Our method, debuting as Inter-MIL, follows a weakly-supervised scheme. It enables the training of the patch-level encoder for WSI in a task-aware optimisation procedure, a step normally not modelled in most existing MIL-based WSI analysis frameworks. We demonstrate that optimising the patch-level encoder is crucial to achieving high-quality fine-grained and tissue-level subtyping results and offers a significant improvement over task-agnostic encoders. Our approach deploys a pseudo-label propagation strategy to update the patch encoder iteratively, allowing discriminative subtype features to be learned. This mechanism also empowers extracting fine-grained attention within image tiles (the small patches), a task largely ignored in most existing weakly supervised-based frameworks. With Inter-MIL, we carried out four challenging cancer molecular subtyping tasks in the context of ovarian, colorectal, lung, and breast cancer. Extensive evaluation results show that Inter-MIL is a robust framework for cancer morpho-molecular subtyping with superior performance compared to several recently proposed methods, in small dataset scenarios where the number of available training slides is less than 100. The iterative optimisation mechanism of Inter-MIL significantly improves the quality of the image features learned by the patch embedded and generally directs the attention map to areas that better align with experts' interpretation, leading to the identification of more reliable histopathology biomarkers. Moreover, an external validation cohort is used to verify the robustness of Inter-MIL on molecular trait prediction.

Authors

  • Yang Hu
    Kweichow Moutai Co., Ltd, Renhuai, Guizhou 564501, China.
  • Korsuk Sirinukunwattana
  • Bin Li
    Department of Magnetic Resonance Imaging (MRI), Beijing Shijitan Hospital, Capital Medical University, Beijing, China.
  • Kezia Gaitskell
    Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of Medicine, University of Oxford, Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
  • Enric Domingo
    Department of Oncology, University of Oxford, Oxford, UK.
  • Willem Bonnaffé
    Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, University of Oxford, Oxford, UK; Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK.
  • Marta Wojciechowska
  • Ruby Wood
    Department of Engineering Science, University of Oxford, Oxford, UK; Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, University of Oxford, Oxford, UK.
  • Nasullah Khalid Alham
    Institute of Biomedical Engineering (IBME), Department of Engineering Science, University of Oxford, Old Road Campus Research Building Oxford, Oxford, UK.
  • Stefano Malacrino
    Institute of Biomedical Engineering (IBME), Department of Engineering Science, University of Oxford, Old Road Campus Research Building Oxford, Oxford, UK.
  • Dan J Woodcock
    Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, University of Oxford, Oxford, UK; Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK.
  • Clare Verrill
    Department of Cellular Pathology, Oxford University Hospitals NHS Foundation Trust, John Radcliffe Hospital, Headley Way, Oxford, OX3 9DU, UK; Nuffield Department of Surgical Sciences, University of Oxford, John Radcliffe Hospital, Headley Way, Oxford, OX3 9DU, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, John Radcliffe Hospital, Headley Way, Oxford, OX3 9DU, UK. Electronic address: Clare.Verrill@ouh.nhs.uk.
  • Ahmed Ahmed
    MRC Weatherall Institute of Molecular Medicine, University of Oxford, Oxford, UK; Nuffield Department of Womenś and Reproductive Health, University of Oxford, Oxford, UK; Oxford National Institute for Health Research (NIHR) Biomedical Research Centre, Oxford, UK.
  • Jens Rittscher
    Department of Engineering Science, University of Oxford, Oxford, United Kingdom.

Keywords

External Resources

Popular Topics
Recent Journals