Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models
Journal:
arXiv
Published Date:
May 11, 2025
Abstract
Vision-Language Models (VLMs) are becoming increasingly popular in the
medical domain, bridging the gap between medical images and clinical language.
Existing VLMs demonstrate an impressive ability to comprehend medical images
and text queries to generate detailed, descriptive diagnostic medical reports.
However, hallucination--the tendency to generate descriptions that are
inconsistent with the visual content--remains a significant issue in VLMs, with
particularly severe implications in the medical field. To facilitate VLM
research on gastrointestinal (GI) image analysis and study hallucination, we
curate a multimodal image-text GI dataset: Gut-VLM. This dataset is created
using a two-stage pipeline: first, descriptive medical reports of Kvasir-v2
images are generated using ChatGPT, which introduces some hallucinated or
incorrect texts. In the second stage, medical experts systematically review
these reports, and identify and correct potential inaccuracies to ensure
high-quality, clinically reliable annotations. Unlike traditional datasets that
contain only descriptive texts, our dataset also features tags identifying
hallucinated sentences and their corresponding corrections. A common approach
to reducing hallucination in VLM is to finetune the model on a small-scale,
problem-specific dataset. However, we take a different strategy using our
dataset. Instead of finetuning the VLM solely for generating textual reports,
we finetune it to detect and correct hallucinations, an approach we call
hallucination-aware finetuning. Our results show that this approach is better
than simply finetuning for descriptive report generation. Additionally, we
conduct an extensive evaluation of state-of-the-art VLMs across several
metrics, establishing a benchmark. GitHub Repo:
https://github.com/bhattarailab/Hallucination-Aware-VLM.
