TTFNet: Temporal-Frequency Features Fusion Network for Speech based Automatic Depression Recognition and Assessment.

Anesthesiology Critical Care Geriatrics Pain Management Primary Care
Journal: IEEE journal of biomedical and health informatics
Published Date: May 29, 2025

Abstract

Related studies have revealed that the phonological features of depressed patients are different from those of healthy individuals. With the increasing prevalence of depression, objective and convenient early screening is necessary. To this end, we propose an automatic depression detection method based on hybrid speech features extracted by deep learning, dubbed as TTFNet. Firstly, to effectively excavate the intrinsic relationship among multidimensional dynamic features in the frequency domain, the Mel spectrogram of raw speech and its related derivatives are encoded into quaternion representation. Then, the innovatively designed quaternion VisionLSTM is utilized to capture their synergistic effects. Simultaneously, we integrate sLSTM with the pre-trained wav2vec 2.0 model to fully acquire the temporal features. In addition, to further exploit the complementarity between temporal and frequency features, we design an XConformer block for cross-sequence interactions, which ingeniously combines self-attention mechanisms and convolutional modules. The designed XCFF fusion module, based on the XConformer block, enables multi-level interactions between frequency-domain and temporal-domain, thereby enhancing generalization ability of the proposed model. Extensive experiments conducted on the AVEC 2013, AVEC 2014, DAIC-WOZ and E-DAIC datasets demonstrate that our method outperforms current state-of-the-art methods in both depression recognition and severity prediction tasks.

Authors

  • Xiyuan Chen
    Department of Mechanical Engineering, Stanford University, Building 530, 440 Escondido Mall, Stanford, CA 94305, USA.
  • Zhuhong Shao
    College of Information Engineering, Capital Normal University, Beijing, 100048, China.
  • Yinan Jiang
  • Runsen Chen
  • Yunlong Wang
    Department of Radiation Oncology, The Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; Guangdong Institute of Gastroenterology, Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, Guangzhou, China.
  • Bicao Li
    School of Electronic and Information Engineering, Zhongyuan University of Technology, Zhengzhou, 450007, China. Electronic address: [email protected].
  • Mingyue Niu
  • Hongguang Chen
  • Qiang Hu
    School of Information Science and Technology, Qingdao University of Science and Technology, Qicngdao 266061, China.
  • Jiasong Wu
    Lab of Image Science and Technology, School of Computer Science and Engineering, Southeast University, Nanjing, China. [email protected].
  • Chunfeng Yang
    Laboratory of Image Science and Technology, Southeast University, Nanjing, Jiangsu 210096, P. R. China.
  • Yuanyuan Shang

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals