RePOPE: Impact of Annotation Errors on the POPE Benchmark
Journal:
arXiv
Published Date:
Apr 22, 2025
Abstract
Since data annotation is costly, benchmark datasets often incorporate labels
from established image datasets. In this work, we assess the impact of label
errors in MSCOCO on the frequently used object hallucination benchmark POPE. We
re-annotate the benchmark images and identify an imbalance in annotation errors
across different subsets. Evaluating multiple models on the revised labels,
which we denote as RePOPE, we observe notable shifts in model rankings,
highlighting the impact of label quality. Code and data are available at
https://github.com/YanNeu/RePOPE .
