R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation
Journal:
arXiv
Published Date:
May 29, 2025
Abstract
Reasoning is a fundamental capability often required in real-world
text-to-image (T2I) generation, e.g., generating ``a bitten apple that has been
left in the air for more than a week`` necessitates understanding temporal
decay and commonsense concepts. While recent T2I models have made impressive
progress in producing photorealistic images, their reasoning capability remains
underdeveloped and insufficiently evaluated. To bridge this gap, we introduce
R2I-Bench, a comprehensive benchmark specifically designed to rigorously assess
reasoning-driven T2I generation. R2I-Bench comprises meticulously curated data
instances, spanning core reasoning categories, including commonsense,
mathematical, logical, compositional, numerical, causal, and concept mixing. To
facilitate fine-grained evaluation, we design R2IScore, a QA-style metric based
on instance-specific, reasoning-oriented evaluation questions that assess three
critical dimensions: text-image alignment, reasoning accuracy, and image
quality. Extensive experiments with 16 representative T2I models, including a
strong pipeline-based framework that decouples reasoning and generation using
the state-of-the-art language and image generation models, demonstrate
consistently limited reasoning performance, highlighting the need for more
robust, reasoning-aware architectures in the next generation of T2I systems.
Project Page: https://r2i-bench.github.io
