VoQA: Visual-only Question Answering
Journal:
arXiv
Published Date:
May 20, 2025
Abstract
We propose Visual-only Question Answering (VoQA), a novel multimodal task in
which questions are visually embedded within images, without any accompanying
textual input. This requires models to locate, recognize, and reason over
visually embedded textual questions, posing challenges for existing large
vision-language models (LVLMs), which show notable performance drops even with
carefully designed prompts. To bridge this gap, we introduce Guided Response
Triggering Supervised Fine-tuning (GRT-SFT), a structured fine-tuning strategy
that guides the model to perform step-by-step reasoning purely based on visual
input, significantly improving model performance. Our work enhances models'
capacity for human-like visual understanding in complex multimodal scenarios,
where information, including language, is perceived visually.
