Dynamic Multi-scale Feature Integration Network for unsupervised MR-CT synthesis.

Journal: Neural networks : the official journal of the International Neural Network Society
Published Date:

Abstract

Unsupervised MR-CT synthesis presents a significant opportunity to reduce radiation exposure from CT scans and lower costs by eliminating the need for both MR and CT imaging. However, many existing unsupervised methods face limitations in capturing different anatomical structures due to their inability to model features with large receptive fields, and the receptive fields of different structures can vary. To address this challenge, we propose a novel Dynamic Multi-scale Feature Integration Network (DMFI-Net) tailored for unsupervised MR-CT synthesis. Our DMFI-Net dynamically adjusts its receptive field to extract multi-scale receptive field features, effectively capturing intricate anatomical details to enhance the synthesis performance. Specifically, we present a Global Context-enhanced Kernel Selection (GCKS) module, which intelligently modulates the receptive fields of convolutions for capturing fine-grained details essential to image transformation. By incorporating global cues, the module enriches multi-scale receptive features with comprehensive semantic information, which is crucial for synthesizing globally distributed target regions or organs. Additionally, a Scale Enhancement Module (SEM) is proposed to integrate features extracted with different scales, preserving richer spatial information. Furthermore, we present a scale-aware reconstruction branch to bolster the encoder's feature extraction capability and improve model generalization. This branch is capable of reconstructing downsampled input images that have undergone random masking, underscoring the model's robust feature extraction ability. Extensive experimental results on one private and two public MR-CT datasets demonstrate that our model significantly outperforms state-of-the-art MR-CT synthesis methods in both qualitative and quantitative evaluations. The implementation code will be released upon acceptance of this manuscript at https://github.com/taozh2017/DMFINet.

Authors

  • Meng Tang
    Department of computer science, University of Waterloo, ON, Canada.
  • Jiuming Jiang
    Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China.
  • Xue Zhang
    School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an 710049, China.
  • Tao Zhou
    Department of Otorhinolaryngology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China.
  • Yizhe Zhang
    Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556.
  • Bin Qiu
    MOE Key Laboratory for Analytical Science of Food Safety and Biology, Fujian Provincial Key Laboratory of Analysis and Detection for Food Safety, College of Chemistry, Fuzhou University, Fuzhou, Fujian 350116, China.
  • Li Zhang
    Department of Animal Nutrition and Feed Science, College of Animal Science and Technology, Huazhong Agricultural University, Wuhan 430070, China.