Visual Abstract Thinking Empowers Multimodal Reasoning
Journal:
arXiv
Published Date:
May 26, 2025
Abstract
Images usually convey richer detail than text, but often include redundant
information which potentially downgrades multimodal reasoning performance. When
faced with lengthy or complex messages, humans tend to employ abstract thinking
to convert them into simple and concise abstracts. Inspired by this cognitive
strategy, we introduce Visual Abstract Thinking (VAT), a novel thinking
paradigm that prompts Multimodal Large Language Models (MLLMs) with visual
abstract instead of explicit verbal thoughts or elaborate guidance, permitting
a more concentrated visual reasoning mechanism. Explicit thinking, such as
Chain-of-thought (CoT) or tool-augmented approaches, increases the complexity
of reasoning process via inserting verbose intermediate steps, external
knowledge or visual information. In contrast, VAT reduces redundant visual
information and encourages models to focus their reasoning on more essential
visual elements. Experimental results show that VAT consistently empowers
different models, and achieves an average gain of 17% over GPT-4o baseline by
employing diverse types of visual abstracts, demonstrating that VAT can enhance
visual reasoning abilities for MLLMs regarding conceptual, structural and
relational reasoning tasks. VAT is also compatible with CoT in
knowledge-intensive multimodal reasoning tasks. These findings highlight the
effectiveness of visual reasoning via abstract thinking and encourage further
exploration of more diverse reasoning paradigms from the perspective of human
cognition.
