Deep Mediation Analysis for Multimodal Genotype-Imaging Associations with Disease Phenotypes

Both genotype and imaging data carry entangled information about the phenotypes. Associations between genotypes and phenotypes can be manifested or absent on biomedical images. While there are abundant multimodal association studies integrating genotype and imaging data, few of them disentangle their direct and indirect effects in associations. We propose GIF (Genotype-Image-Phenotype), a novel modeling framework to predict phenotypes by genotype and imaging data with a mediation structure of these three sets of variables. GIF constitutes a backbone of mediation path genotype→image→phenotype, and direct genotype→phenotype and image→phenotype paths capturing the additional direct associations not explained by the mediation model. To implement the mediation model with a convolutional neural network (CNN), we constructed a CNN to predict genotypes with images and then employed the joint embedding vectors of the CNN to predict phenotypes. The direct links were sequentially augmented to the mediation model to further improve prediction accuracy. We validated GIF on three datasets: (1) a synthetic polygon dataset where the presence or absence of a polygon specie indicates a genotype and selected combinations of multiple polygon species indicate phenotypes, (2) the PASCAL VOL dataset of object detection and action recognition, where the presence or absence of an object class indicates a genotype and the presence or absence of an action class indicates a phenotype, (3) the ADNI dataset of Alzheimer’s disease diagnosis comprising the genotype and imaging data of subjects with three cognitive states. GIF prediction outcomes on the ADNI data indicate the associations between genotypes and phenotypes are primarily mediated by image features, and images provide additional information about phenotypes which is not attributed to genotype variations.