228 × 304 200-mW lidar based on a single-point global-depth d-ToF sensor and RGB-guided super-resolution neural network.

Journal: Optics letters
Published Date: Jul 1, 2023

Abstract

The cutting-edge imaging system exhibits low output resolution and high power consumption, presenting challenges for the RGB-D fusion algorithm. In practical scenarios, aligning the depth map resolution with the RGB image sensor is a crucial requirement. In this Letter, the software and hardware co-design is considered to implement a lidar system based on the monocular RGB 3D imaging algorithm. A 6.4 × 6.4-mm deep-learning accelerator (DLA) system-on-chip (SoC) manufactured in a 40-nm CMOS is incorporated with a 3.6-mm TX-RX integrated chip fabricated in a 180-nm CMOS to employ the customized single-pixel imaging neural network. In comparison to the RGB-only monocular depth estimation technique, the root mean square error is reduced from 0.48 m to 0.3 m on the evaluated dataset, and the output depth map resolution matches the RGB input.

Authors

  • Miao Sun
    Department of Electrical and Computer Engineering.
  • Yifan Wu
    Department of Information Science and Technology, Northwest University, Xi'an, Shaanxi 710127, China.
  • Lebei Cui
  • Hengwei Yu
  • Jie Li
    Guangdong-Hong Kong-Macao Greater Bay Area Artificial Intelligence Application Technology Research Institute, Shenzhen Polytechnic University, Shenzhen, China.
  • Jian Qian
    Department of Urology, the First Affiliated Hospital of Nanjing Medical University, Nanjing, China.
  • Jier Wang
  • Lei Qiu
    Department of Gastric Surgery, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Affiliated Cancer Hospital of University of Electronic Science and Technology of China, Chengdu, China.
  • Patrick Yin Chiang
  • Shenglong Zhuo
    State Key Laboratory of ASIC and System, Fudan University, No. 825, Zhangheng Road, Shanghai 201203, China.

Keywords

External Resources

Popular Topics
Recent Journals