Attention-Guided Integration of CLIP and SAM for Precise Object Masking in Robotic Manipulation
Journal:
arXiv
Published Date:
Feb 26, 2025
Abstract
This paper introduces a novel pipeline to enhance the precision of object
masking for robotic manipulation within the specific domain of masking products
in convenience stores. The approach integrates two advanced AI models, CLIP and
SAM, focusing on their synergistic combination and the effective use of
multimodal data (image and text). Emphasis is placed on utilizing
gradient-based attention mechanisms and customized datasets to fine-tune
performance. While CLIP, SAM, and Grad- CAM are established components, their
integration within this structured pipeline represents a significant
contribution to the field. The resulting segmented masks, generated through
this combined approach, can be effectively utilized as inputs for robotic
systems, enabling more precise and adaptive object manipulation in the context
of convenience store products.
