Touchless interactive teaching of soft robots through flexible bimodal sensory interfaces.

Journal: Nature communications
Published Date: Aug 26, 2022

Abstract

In this paper, we propose a multimodal flexible sensory interface for interactively teaching soft robots to perform skilled locomotion using bare human hands. First, we develop a flexible bimodal smart skin (FBSS) based on triboelectric nanogenerator and liquid metal sensing that can perform simultaneous tactile and touchless sensing and distinguish these two modes in real time. With the FBSS, soft robots can react on their own to tactile and touchless stimuli. We then propose a distance control method that enabled humans to teach soft robots movements via bare hand-eye coordination. The results showed that participants can effectively teach a self-reacting soft continuum manipulator complex motions in three-dimensional space through a "shifting sensors and teaching" method within just a few minutes. The soft manipulator can repeat the human-taught motions and replay them at different speeds. Finally, we demonstrate that humans can easily teach the soft manipulator to complete specific tasks such as completing a pen-and-paper maze, taking a throat swab, and crossing a barrier to grasp an object. We envision that this user-friendly, non-programmable teaching method based on flexible multimodal sensory interfaces could broadly expand the domains in which humans interact with and utilize soft robots.

Authors

  • Wenbo Liu
    College of Environment and Plant Protection, Hainan University, Haikou 570228, China.
  • Youning Duo
    School of Mechanical Engineering and Automation, Beihang University, Beijing, 100191, China.
  • Jiaqi Liu
  • Feiyang Yuan
    School of Mechanical Engineering and Automation, Beihang University, Beijing, 100191, China.
  • Lei Li
    Department of Thoracic Surgery, The Affiliated Huaian No.1 People's Hospital of Nanjing Medical University, Huai'an, China.
  • Luchen Li
    School of Mechanical Engineering and Automation, Beihang University, Beijing, 100191, China.
  • Gang Wang
    National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China.
  • Bohan Chen
  • Siqi Wang
    School of Mechanical Engineering and Automation, Beihang University, Beijing, 100191, People's Republic of China.
  • Hui Yang
    Department of Neurology, The Second Affiliated Hospital of Guizhou University of Chinese Medicine, Guiyang, China.
  • Yuchen Liu
    Department of Internal Medicine, Peking Union Medical College Hospital, Beijing, China.
  • Yanru Mo
    School of General Engineering, Beihang University, Beijing, 100191, China.
  • Yun Wang
    Department of Anesthesiology, Beijing Friendship Hospital, Capital Medical University, Beijing, 100050, People's Republic of China.
  • Bin Fang
    Department of Earth and Environmental Engineering, Columbia University, New York, 10027, USA.
  • Fuchun Sun
    Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China.
  • Xilun Ding
    School of Mechanical Engineering and Automation, Beihang University, Beijing 100191, China.
  • Chi Zhang
    Department of Thoracic Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China.
  • Li Wen

Keywords

External Resources

Popular Topics
Recent Journals