Look-to-Touch: A Vision-Enhanced Proximity and Tactile Sensor for Distance and Geometry Perception in Robotic Manipulation
Journal:
arXiv
Published Date:
Apr 14, 2025
Abstract
Camera-based tactile sensors provide robots with a high-performance tactile
sensing approach for environment perception and dexterous manipulation.
However, achieving comprehensive environmental perception still requires
cooperation with additional sensors, which makes the system bulky and limits
its adaptability to unstructured environments. In this work, we present a
vision-enhanced camera-based dual-modality sensor, which realizes full-scale
distance sensing from 50 cm to -3 mm while simultaneously keeping
ultra-high-resolution texture sensing and reconstruction capabilities. Unlike
conventional designs with fixed opaque gel layers, our sensor features a
partially transparent sliding window, enabling mechanical switching between
tactile and visual modes. For each sensing mode, a dynamic distance sensing
model and a contact geometry reconstruction model are proposed. Through
integration with soft robotic fingers, we systematically evaluate the
performance of each mode, as well as in their synergistic operation.
Experimental results show robust distance tracking across various speeds,
nanometer-scale roughness detection, and sub-millimeter 3D texture
reconstruction. The combination of both modalities improves the robot's
efficiency in executing grasping tasks. Furthermore, the embedded mechanical
transmission in the sensor allows for fine-grained intra-hand adjustments and
precise manipulation, unlocking new capabilities for soft robotic hands.
