BBNet: A Novel Convolutional Neural Network Structure in Edge-Cloud Collaborative Inference.

Journal: Sensors (Basel, Switzerland)
Published Date:

Abstract

Edge-cloud collaborative inference can significantly reduce the delay of a deep neural network (DNN) by dividing the network between mobile edge and cloud. However, the in-layer data size of DNN is usually larger than the original data, so the communication time to send intermediate data to the cloud will also increase end-to-end latency. To cope with these challenges, this paper proposes a novel convolutional neural network structure-BBNet-that accelerates collaborative inference from two levels: (1) through channel-pruning: reducing the number of calculations and parameters of the original network; (2) through compressing the feature map at the split point to further reduce the size of the data transmitted. In addition, This paper implemented the BBNet structure based on NVIDIA Nano and the server. Compared with the original network, BBNet's FLOPs and parameter achieve up to 5.67× and 11.57× on the compression rate, respectively. In the best case, the feature compression layer can reach a bit-compression rate of 512×. Compared with the better bandwidth conditions, BBNet has a more obvious inference delay when the network conditions are poor. For example, when the upload bandwidth is only 20 kb/s, the end-to-end latency of BBNet is increased by 38.89× compared with the cloud-only approach.

Authors

  • Hongbo Zhou
  • Weiwei Zhang
    Department of Laboratory Medicine, Affiliated Hospital of Nantong University, Nantong, Jiangsu, China.
  • Chengwei Wang
    Department of Neurosurgery, the Second Hospital of Shandong University, Jinan, China.
  • Xin Ma
    Department of Medical Oncology, Harbin Medical University Cancer Hospital, Harbin, China.
  • Haoran Yu
    College of Veterinary Medicine, Northeast Agricultural University, Harbin 150030, China; Key Laboratory of the Provincial Education, Department of Heilongjiang for Common Animal Disease Prevention and Treatment, Northeast Agricultural University, Harbin 150030, China.