Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model
Journal:
arXiv
Published Date:
Feb 13, 2025
Abstract
We propose general visual inspection model using Vision-Language Model~(VLM)
with few-shot images of non-defective or defective products, along with
explanatory texts that serve as inspection criteria. Although existing VLM
exhibit high performance across various tasks, they are not trained on specific
tasks such as visual inspection. Thus, we construct a dataset consisting of
diverse images of non-defective and defective products collected from the web,
along with unified formatted output text, and fine-tune VLM. For new products,
our method employs In-Context Learning, which allows the model to perform
inspections with an example of non-defective or defective image and the
corresponding explanatory texts with visual prompts. This approach eliminates
the need to collect a large number of training samples and re-train the model
for each product. The experimental results show that our method achieves high
performance, with MCC of 0.804 and F1-score of 0.950 on MVTec AD in a one-shot
manner. Our code is available
at~https://github.com/ia-gu/Vision-Language-In-Context-Learning-Driven-Few-Shot-Visual-Inspection-Model.
