An artificial visual neuron with multiplexed rate and time-to-first-spike coding.

Journal: Nature communications
Published Date: May 1, 2024

Abstract

Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial visual neurons for use in spiking neural network (SNN). However, the lack of multiplexed data coding schemes reduces the ability of artificial visual neurons in SNN to emulate the visual perception ability of biological systems. Here, we present an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The artificial neuron can code visual information at different spiking frequencies (rate coding) and enables precise and energy-efficient time-to-first-spike (TTFS) coding. This multiplexed sensory coding scheme could improve the computing capability and efficacy of artificial visual neurons. A hardware-based SNN with the RTF coding scheme exhibits good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The multiplexed RTF coding scheme demonstrates the feasibility of developing highly efficient spike-based neuromorphic hardware.

Authors

  • Fanfan Li
    School of Materials Science and Engineering, Zhejiang University, Hangzhou, China.
  • Dingwei Li
    Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China.
  • Chuanqing Wang
    CenBRAIN Neurotech, School of Engineering, Westlake University, Hangzhou, China.
  • Guolei Liu
    Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China.
  • Rui Wang
    Department of Clinical Laboratory Medicine Center, Inner Mongolia Autonomous Region People's Hospital, Hohhot, Inner Mongolia, China.
  • Huihui Ren
    School of Information Science and Engineering, Yanshan University, Hebei Avenue, Qinhuangdao, 066004, China.
  • Yingjie Tang
    Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China.
  • Yan Wang
    College of Animal Science and Technology, Beijing University of Agriculture, Beijing, China.
  • Yitong Chen
    Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China.
  • Kun Liang
    Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, N2L 3G1, Canada. kun.liang@uwaterloo.ca.
  • Qi Huang
    State Key Laboratory of Agricultural Microbiology, College of Veterinary Medicine, Huazhong Agricultural Universitygrid.35155.37, Wuhan, China.
  • Mohamad Sawan
    CenBRAIN Neurotech Center of Excellence, School of Engineering, Westlake University, Hangzhou, China.
  • Min Qiu
    Affiliated Hospital of Jining Medical University, Jining 272029, China.
  • Hong Wang
    Department of Cardiology, Liuzhou Workers' Hospital, The Fourth Affiliated Hospital of Guangxi Medical University, Liuzhou, China.
  • Bowen Zhu
    Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China. zhubowen@westlake.edu.cn.

Keywords

External Resources

Popular Topics
Recent Journals