An Attentive Dual-Encoder Framework Leveraging Multimodal Visual and Semantic Information for Automatic OSAHS Diagnosis
Journal:
arXiv
Published Date:
Dec 25, 2024
Abstract
Obstructive sleep apnea-hypopnea syndrome (OSAHS) is a common sleep disorder
caused by upper airway blockage, leading to oxygen deprivation and disrupted
sleep. Traditional diagnosis using polysomnography (PSG) is expensive,
time-consuming, and uncomfortable. Existing deep learning methods using facial
image analysis lack accuracy due to poor facial feature capture and limited
sample sizes. To address this, we propose a multimodal dual encoder model that
integrates visual and language inputs for automated OSAHS diagnosis. The model
balances data using randomOverSampler, extracts key facial features with
attention grids, and converts physiological data into meaningful text.
Cross-attention combines image and text data for better feature extraction, and
ordered regression loss ensures stable learning. Our approach improves
diagnostic efficiency and accuracy, achieving 91.3% top-1 accuracy in a
four-class severity classification task, demonstrating state-of-the-art
performance. Code will be released upon acceptance.
