SegSub: Evaluating Robustness to Knowledge Conflicts and Hallucinations in Vision-Language Models
Journal:
arXiv
Published Date:
Feb 19, 2025
Abstract
Vision language models (VLM) demonstrate sophisticated multimodal reasoning
yet are prone to hallucination when confronted with knowledge conflicts,
impeding their deployment in information-sensitive contexts. While existing
research addresses robustness in unimodal models, the multimodal domain lacks
systematic investigation of cross-modal knowledge conflicts. This research
introduces \segsub, a framework for applying targeted image perturbations to
investigate VLM resilience against knowledge conflicts. Our analysis reveals
distinct vulnerability patterns: while VLMs are robust to parametric conflicts
(20% adherence rates), they exhibit significant weaknesses in identifying
counterfactual conditions (<30% accuracy) and resolving source conflicts (<1%
accuracy). Correlations between contextual richness and hallucination rate (r =
-0.368, p = 0.003) reveal the kinds of images that are likely to cause
hallucinations. Through targeted fine-tuning on our benchmark dataset, we
demonstrate improvements in VLM knowledge conflict detection, establishing a
foundation for developing hallucination-resilient multimodal systems in
information-sensitive environments.
