DeFusion: An Effective Decoupling Fusion Network for Multi-Modal Pregnancy Prediction
Journal:
arXiv
Published Date:
Jan 8, 2025
Abstract
Temporal embryo images and parental fertility table indicators are both
valuable for pregnancy prediction in \textbf{in vitro fertilization embryo
transfer} (IVF-ET). However, current machine learning models cannot make full
use of the complementary information between the two modalities to improve
pregnancy prediction performance. In this paper, we propose a Decoupling Fusion
Network called DeFusion to effectively integrate the multi-modal information
for IVF-ET pregnancy prediction. Specifically, we propose a decoupling fusion
module that decouples the information from the different modalities into
related and unrelated information, thereby achieving a more delicate fusion.
And we fuse temporal embryo images with a spatial-temporal position encoding,
and extract fertility table indicator information with a table transformer. To
evaluate the effectiveness of our model, we use a new dataset including 4046
cases collected from Southern Medical University. The experiments show that our
model outperforms state-of-the-art methods. Meanwhile, the performance on the
eye disease prediction dataset reflects the model's good generalization. Our
code and dataset are available at https://github.com/Ou-Young-1999/DFNet.
