Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers
Journal:
arXiv
Published Date:
Jun 30, 2025
Abstract
Recent progress in multimodal reasoning has been significantly advanced by
textual Chain-of-Thought (CoT), a paradigm where models conduct reasoning
within language. This text-centric approach, however, treats vision as a
static, initial context, creating a fundamental "semantic gap" between rich
perceptual data and discrete symbolic thought. Human cognition often transcends
language, utilizing vision as a dynamic mental sketchpad. A similar evolution
is now unfolding in AI, marking a fundamental paradigm shift from models that
merely think about images to those that can truly think with images. This
emerging paradigm is characterized by models leveraging visual information as
intermediate steps in their thought process, transforming vision from a passive
input into a dynamic, manipulable cognitive workspace. In this survey, we chart
this evolution of intelligence along a trajectory of increasing cognitive
autonomy, which unfolds across three key stages: from external tool
exploration, through programmatic manipulation, to intrinsic imagination. To
structure this rapidly evolving field, our survey makes four key contributions.
(1) We establish the foundational principles of the think with image paradigm
and its three-stage framework. (2) We provide a comprehensive review of the
core methods that characterize each stage of this roadmap. (3) We analyze the
critical landscape of evaluation benchmarks and transformative applications.
(4) We identify significant challenges and outline promising future directions.
By providing this structured overview, we aim to offer a clear roadmap for
future research towards more powerful and human-aligned multimodal AI.
