Real-time fine finger motion decoding for transradial amputees with surface electromyography.

Journal: Neural networks : the official journal of the International Neural Network Society
Published Date: May 27, 2025

Abstract

Advancements in human-machine interfaces (HMIs) are pivotal for enhancing rehabilitation technologies and improving the quality of life for individuals with limb loss. This paper presents a novel CNN-Transformer model for decoding continuous fine finger motions from surface electromyography (sEMG) signals by integrating the convolutional neural network (CNN) and Transformer architecture, focusing on applications for transradial amputees. This model leverages the strengths of both convolutional and Transformer architectures to effectively capture both local muscle activation patterns and global temporal dependencies within sEMG signals. To achieve high-fidelity sEMG acquisition, we designed a flexible and stretchable epidermal array electrode sleeve (EAES) that conforms to the residual limb, ensuring comfortable long-term wear and robust signal capture, critical for amputees. Moreover, we presented a computer vision (CV) based multimodal data acquisition protocol that synchronizes sEMG recordings with video captures of finger movements, enabling the creation of a large, labeled dataset to train and evaluate the proposed model. Given the challenges in acquiring reliable labeled data for transradial amputees, we adopted transfer learning and few-shot calibration to achieve fine finger motion decoding by leveraging datasets from non-amputated subjects. Extensive experimental results demonstrate the superior performance of the proposed model in various scenarios, including intra-session, inter-session, and inter-subject evaluations. Importantly, the system also exhibited promising zero-shot and few-shot learning capabilities for amputees, allowing for personalized calibration with minimal training data. The combined approach holds significant potential for advancing real-time, intuitive control of prostheses and other assistive technologies.

Authors

  • Zihan Weng
    Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, 611731, Chengdu, China; School of Life Science and Technology, Center for Information in Medicine, University of Electronic Science and Technology of China, 611731, Chengdu, China.
  • Yang Xiao
  • Peiyang Li
    Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, 610054, China.
  • Chanlin Yi
    The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for NeuroInformation, University of Electronic Science and Technology of China, Chengdu, 611731, China; School of Life Science and Technology, Center for Information in Medicine, University of Electronic Science and Technology of China, Chengdu, 611731, China.
  • Pouya Bashivan
    McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139.
  • Hailin Ma
    Plateau Brain Science Research Center, South China Normal University/Tibet University, 850012, Lhasa, China.
  • Guang Yao
    Department of Molecular and Cellular Biology, University of Arizona, Tucson, AZ 85721, USA.
  • Yuan Lin
  • Fali Li
    The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Chengdu, 611731, China; School of Life Science and Technology, Center for Information in Medicine, University of Electronic Science and Technology of China, Chengdu, 611731, China.
  • DeZhong Yao
    The Key Laboratory for Neuro Information of Ministry of Education, Center for Information in Bio Medicine, High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China.
  • Jingming Hou
    Center for Artificial Intelligence Technology, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, Malaysia. Electronic address: p138473@siswa.ukm.edu.my.
  • Yangsong Zhang
  • Peng Xu
    Department of Urology, Zhujiang Hospital, Southern Medical University, Guangzhou, China.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals