Machine learning-assisted wearable sensing systems for speech recognition and interaction.

Journal: Nature communications
PMID: 40064879

Abstract

The human voice stands out for its rich information transmission capabilities. However, voice communication is susceptible to interference from noisy environments and obstacles. Here, we propose a wearable wireless flexible skin-attached acoustic sensor (SAAS) capable of capturing the vibrations of vocal organs and skin movements, thereby enabling voice recognition and human-machine interaction (HMI) in harsh acoustic environments. This system utilizes a piezoelectric micromachined ultrasonic transducers (PMUT), which feature high sensitivity (-198 dB), wide bandwidth (10 Hz-20 kHz), and excellent flatness (±0.5 dB). Flexible packaging enhances comfort and adaptability during wear, while integration with the Residual Network (ResNet) architecture significantly improves the classification of laryngeal speech features, achieving an accuracy exceeding 96%. Furthermore, we also demonstrated SAAS's data collection and intelligent classification capabilities in multiple HMI scenarios. Finally, the speech recognition system was able to recognize everyday sentences spoken by participants with an accuracy of 99.8% through a deep learning model. With advantages including a simple fabrication process, stable performance, easy integration, and low cost, SAAS presents a compelling solution for applications in voice control, HMI, and wearable electronics.

Authors

  • Tao Liu
    Institute of Urology and Nephrology, The First Affiliated Hospital of Guangxi Medical University, Nanning, China.
  • Mingyang Zhang
    School of Quality and Technical Supervision, Hebei University, Baoding, Hebei 071002, P.R.China.
  • Zhihao Li
    Heilongjiang University of CM, Harbin 150040, China.
  • Hanjie Dou
    Key Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing University, Chongqing, 400044, China.
  • Wangyang Zhang
    Key Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing University, Chongqing, 400044, China.
  • Jiaqian Yang
    College of Environment, Zhejiang University of Technology, Hangzhou 310032, China.
  • Pengfan Wu
    Key Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing University, Chongqing, 400044, China.
  • Dongxiao Li
    Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583, Singapore. elelc@nus.edu.sg.
  • Xiaojing Mu
    Key Disciplines Lab of Novel Micro-nano Devices and System Technology, Chongqing University, Chongqing 400030, China; Key Laboratory for Optoelectronic Technology & System of Ministry of Education, Chongqing University, Chongqing 400044, China; International R & D center of Micro-nano Systems and New Materials Technology, Chongqing University, Chongqing 400030, China; School of Optoelectronics Engineering, Chongqing University, Chongqing 400044, China.

Keywords

External Resources

Popular Topics
Recent Journals