Universal encoding of pan-cancer histology by deep texture representations.

Journal: Cell reports
Published Date: Mar 1, 2022

Abstract

Cancer histological images contain rich biological and clinical information, but quantitative representation can be problematic and has prevented the direct comparison and accumulation of large-scale datasets. Here, we show successful universal encoding of cancer histology by deep texture representations (DTRs) produced by a bilinear convolutional neural network. DTR-based, unsupervised histological profiling, which captures the morphological diversity, is applied to cancer biopsies and reveals relationships between histologic characteristics and the response to immune checkpoint inhibitors (ICIs). Content-based image retrieval based on DTRs enables the quick retrieval of histologically similar images using The Cancer Genome Atlas (TCGA) dataset. Furthermore, via comprehensive comparisons with driver and clinically actionable gene mutations, we successfully predict 309 combinations of genomic features and cancer types from hematoxylin-and-eosin-stained images. With its mounting capabilities on accessible devices, such as smartphones, universal encoding for cancer histology has a strong impact on global equalization for cancer diagnosis and therapies.

Authors

  • Daisuke Komura
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
  • Akihiro Kawabe
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan; Department of Pathology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Keisuke Fukuta
    Department of Genomic Pathology, Medical Research Institute, Tokyo Medical and Dental University, 1-5-45 Yushima, Bunkyo-ku, Tokyo 1138510, Japan.
  • Kyohei Sano
    Department of Cognitive Behavioral Physiology, Chiba University Graduate School of Medicine, 1-8-1 Inohana, Chiba, Chiba, 260-8670, Japan.
  • Toshikazu Umezaki
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Hirotomo Koda
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
  • Ryohei Suzuki
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Ken Tominaga
    Department of Genomic Pathology, Medical Research Institute, Tokyo Medical and Dental University, 1-5-45 Yushima, Bunkyo-ku, Tokyo 1138510, Japan.
  • Mieko Ochi
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Hiroki Konishi
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
  • Fumiya Masakado
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Noriyuki Saito
    Department of Gastrointestinal Surgery, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Yasuyoshi Sato
    Department of Gastrointestinal Surgery, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Takumi Onoyama
    Division of Gastroenterology and Nephrology, Department of Multidisciplinary Internal Medicine, Tottori University Faculty of Medicine, Yonago, Japan.
  • Shu Nishida
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan; Department of Pathology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Genta Furuya
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan; Department of Pathology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Hiroto Katoh
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
  • Hiroharu Yamashita
    Department of Gastrointestinal Surgery, Graduate School of Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan.
  • Kazuhiro Kakimi
    Department of Immunotherapeutics, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo 1130033, Japan.
  • Yasuyuki Seto
    Department of Gastrointestinal Surgery, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
  • Tetsuo Ushiku
    Department of Pathology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan. usikut-tky@umin.ac.jp.
  • Masashi Fukayama
    Department of Pathology, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
  • Shumpei Ishikawa
    Department of Preventive Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan. ishum-prm@m.u-tokyo.ac.jp.

Keywords

External Resources

Popular Topics
Recent Journals