Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces.

Journal: Nature communications
Published Date: Oct 3, 2022

Abstract

A wearable silent speech interface (SSI) is a promising platform that enables verbal communication without vocalization. The most widely studied methodology for SSI focuses on surface electromyography (sEMG). However, sEMG suffers from low scalability because of signal quality-related issues, including signal-to-noise ratio and interelectrode interference. Hence, here, we present a novel SSI by utilizing crystalline-silicon-based strain sensors combined with a 3D convolutional deep learning algorithm. Two perpendicularly placed strain gauges with minimized cell dimension (<0.1 mm) could effectively capture the biaxial strain information with high reliability. We attached four strain sensors near the subject's mouths and collected strain data of unprecedently large wordsets (100 words), which our SSI can classify at a high accuracy rate (87.53%). Several analysis methods were demonstrated to verify the system's reliability, as well as the performance comparison with another SSI using sEMG electrodes with the same dimension, which exhibited a relatively low accuracy rate (42.60%).

Authors

  • Taemin Kim
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Yejee Shin
    Medical Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Kyowon Kang
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Kiho Kim
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Gwanho Kim
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Yunsu Byeon
    Medical Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Hwayeon Kim
    Digital Signal Processing & Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Yuyan Gao
    Department of Engineering Science and Mechanics, The Pennsylvania State University, University Park, PA, 16802, USA.
  • Jeong Ryong Lee
    Medical Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Geonhui Son
    Medical Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Taeseong Kim
    School of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea.
  • Yohan Jun
    School of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea.
  • Jihyun Kim
    Quality Evaluation Team, Samsung Bioepis Co., Ltd, Incheon, Republic of Korea.
  • Jinyoung Lee
    Digital Signal Processing & Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Seyun Um
    Digital Signal Processing & Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Yoohwan Kwon
    Digital Signal Processing & Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Byung Gwan Son
    Digital Signal Processing & Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Myeongki Cho
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Mingyu Sang
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Jongwoon Shin
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Kyubeen Kim
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Jungmin Suh
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Heekyeong Choi
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Seokjun Hong
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea.
  • Huanyu Cheng
    Department of Engineering Science and Mechanics, The Pennsylvania State University, University Park, PA, 16802, USA.
  • Hong-Goo Kang
    Digital Signal Processing & Artificial Intelligence Lab, School of Electrical and Electronic Engineering, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea. hgkang@yonsei.ac.kr.
  • Dosik Hwang
    School of Electrical and Electronic Engineering, Yonsei University, Seoul, Republic of Korea. dosik.hwang@yonsei.ac.kr.
  • Ki Jun Yu
    Functional Bio-integrated Electronics and Energy Management Lab, School of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea. kijunyu@yonsei.ac.kr.

Keywords

External Resources

Popular Topics
Recent Journals