A visual-omics foundation model to bridge histopathology with spatial transcriptomics.

Journal: Nature methods
Published Date: May 29, 2025

Abstract

Artificial intelligence has revolutionized computational biology. Recent developments in omics technologies, including single-cell RNA sequencing and spatial transcriptomics, provide detailed genomic data alongside tissue histology. However, current computational models focus on either omics or image analysis, lacking their integration. To address this, we developed OmiCLIP, a visual-omics foundation model linking hematoxylin and eosin images and transcriptomics using tissue patches from Visium data. We transformed transcriptomic data into 'sentences' by concatenating top-expressed gene symbols from each patch. We curated a dataset of 2.2 million paired tissue images and transcriptomic data across 32 organs to train OmiCLIP integrating histology and transcriptomics. Building on OmiCLIP, our Loki platform offers five key functions: tissue alignment, annotation via bulk RNA sequencing or marker genes, cell-type decomposition, image-transcriptomics retrieval and spatial transcriptomics gene expression prediction from hematoxylin and eosin-stained images. Compared with 22 state-of-the-art models on 5 simulations, and 19 public and 4 in-house experimental datasets, Loki demonstrated consistent accuracy and robustness.

Authors

  • Weiqing Chen
    Department of Epidemiology and Health Statistics, School of Public Health, Sun Yat-Sen University, Guangzhou, 510080, China. chenwq@mail.sysu.edu.cn.
  • Pengzhi Zhang
    Center for Bioinformatics and Computational Biology, Houston Methodist Research Institute, Houston, Texas, USA.
  • Tu N Tran
    Center for Bioinformatics and Computational Biology, Houston Methodist Research Institute, Houston, TX, USA.
  • Yiwei Xiao
    Center for Bioinformatics and Computational Biology, Houston Methodist Research Institute, Houston, TX, USA.
  • Shengyu Li
    School of Computing, University of South Alabama, Mobile, AL, 36688, USA.
  • Vrutant V Shah
    Center for RNA Therapeutics, Houston Methodist Research Institute, Houston, TX, USA.
  • Hao Cheng
    Department of Forensic Pathology, School of Forensic Medicine, China Medical University, Shenyang 110122, China.
  • Kristopher W Brannan
    Department of Cellular and Molecular Medicine, University of Califorinia, San Diego, La Jolla, CA, USA; Institute for Genomic Medicine and UCSD Stem Cell Program, University of California, San Diego, La Jolla, CA, USA; Stem Cell Program, University of California, San Diego, La Jolla, CA, USA.
  • Keith Youker
    Center for Cardiovascular Regeneration, Houston Methodist Research Institute, Houston, TX, USA.
  • Li Lai
    Center for Cardiovascular Regeneration, Houston Methodist Research Institute, Houston, TX, USA.
  • Longhou Fang
    Center for Cardiovascular Regeneration, Houston Methodist Research Institute, Houston, TX, USA.
  • Yu Yang
    Department of Obstetrics & Gynecology, the First Affiliated Hospital of Xi'an Jiaotong University, Xian, Shaanxi, China.
  • Nhat-Tu Le
    Center for Cardiovascular Regeneration, Houston Methodist Research Institute, Houston, TX, USA.
  • Jun-Ichi Abe
    Department of Cardiology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
  • Shu-Hsia Chen
    Center for Immunotherapy, Neal Cancer Center, Houston Methodist Research Institute, Houston, TX, USA.
  • Qin Ma
    Computational Systems Biology Lab, Department of Biochemistry and Molecular Biology, and Institute of Bioinformatics, University of Georgia, GA 30602, USA BioEnergy Science Center, TN 37831, USA.
  • Ken Chen
    The Genome Institute, Washington University in St. Louis, St. Louis, Missouri, United States of America.
  • Qianqian Song
    Wake Forest School of Medicine.
  • John P Cooke
    Department of Cardiovascular Sciences, Houston Methodist Research Institute, Houston, Tex; Center for Cardiovascular Regeneration, Houston Methodist DeBakey Heart and Vascular Center, Houston, Tex.
  • Guangyu Wang
    State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals