Enhanced diabetic retinopathy detection via multimodal fusion of fundus imaging features and clinical demographic information using verifiable convolutional neural network.

Traditional systems of ocular disease diagnosis and many deep learning-based systems are limited in their practice to analyzing fundus images from a single eye, often using raw, unprocessed data. This fails to include the critical correlation between bilateral eyes, has several issues such as low image contrast, noise, and class imbalance, and generally concentrates on the detection of no more than a single disease. In order to bridge the gap, Enhanced Diabetic Retinopathy Detection through Multimodal Integration of Fundus Imaging Features and Clinical Demographic Information utilizing Verifiable Convolutional Neural Network (DD-FIF-CDI-VCNN) is proposed. In the initial phase, the fundus images of the eyes along with their corresponding demographic information are sourced from the Ocular Disease Recognition, which acts as the main feed for the proposed framework. During the pre-processing stage, it uses Robust Consensus Tobit Kalman Filtering (RCTKF) to resize the fundus images, normalizes the pixel intensities, and reduces noise in the process of quality improvement for subsequent feature extraction. The pre-processed fundus images are provided to the ResNet-fused External Attention Network (ResfEANet) based feature extraction that extracts rich visual and texture features from the retinal images. In parallel to this, the demographic attributes are fed to a feature extraction module using TabNet which converts the structured affected person facts into a meaningful function vector. This aims at making sure that relevant patient-specific styles are encoded and prepared for integration with visual functions. Then, the visible and texture capabilities extracted from the fundus photos are combined with the demographic attributes the usage of Hierarchical Multi-Scale Feature Fusion (HMSFF). This creates a complete representation that leverages both imaging and structured affected person statistics. This blended feature set is the input to the categorization model for correct diabetic retinopathy detection. Next, this fused characteristic vector, combining the features extracted from the fundus images with their corresponding demographic attributes, is passed to a Verifiable Convolutional Neural Network (VCNN)-based classifier model. This comprehensive multi-modal representation ensues, rendering the VCNN model capable of accurate detection of diabetic retinopathy classifying normal and diabetes, thereby enhancing predictability and robustness compared with the unimodal approach. The VCNN parameters are fine-tuned by the Warthog Optimization Algorithm (WOA) to ensure improved convergence and better detection performance. The proposed DD-FIF-CDI-VCNN method is analyzed under performance metrics: accuracy, precision, recall, F1-Score, Area under Curve (AUC) and Error rate when compared with existing models.