Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Models
Journal:
arXiv
Published Date:
May 20, 2025
Abstract
Additive manufacturing enables the fabrication of complex designs while
minimizing waste, but faces challenges related to defects and process
anomalies. This study presents a novel multimodal Retrieval-Augmented
Generation-based framework that automates anomaly detection across various
Additive Manufacturing processes leveraging retrieved information from
literature, including images and descriptive text, rather than training
datasets. This framework integrates text and image retrieval from scientific
literature and multimodal generation models to perform zero-shot anomaly
identification, classification, and explanation generation in a Laser Powder
Bed Fusion setting. The proposed framework is evaluated on four L-PBF
manufacturing datasets from Oak Ridge National Laboratory, featuring various
printer makes, models, and materials. This evaluation demonstrates the
framework's adaptability and generalizability across diverse images without
requiring additional training. Comparative analysis using Qwen2-VL-2B and
GPT-4o-mini as MLLM within the proposed framework highlights that GPT-4o-mini
outperforms Qwen2-VL-2B and proportional random baseline in manufacturing
anomalies classification. Additionally, the evaluation of the RAG system
confirms that incorporating retrieval mechanisms improves average accuracy by
12% by reducing the risk of hallucination and providing additional information.
The proposed framework can be continuously updated by integrating emerging
research, allowing seamless adaptation to the evolving landscape of AM
technologies. This scalable, automated, and zero-shot-capable framework
streamlines AM anomaly analysis, enhancing efficiency and accuracy.
