Demystifying the Visual Quality Paradox in Multimodal Large Language Models
Journal:
arXiv
Published Date:
Jun 18, 2025
Abstract
Recent Multimodal Large Language Models (MLLMs) excel on benchmark
vision-language tasks, yet little is known about how input visual quality
shapes their responses. Does higher perceptual quality of images already
translate to better MLLM understanding? We conduct the first systematic study
spanning leading MLLMs and a suite of vision-language benchmarks, applying
controlled degradations and stylistic shifts to each image. Surprisingly, we
uncover a visual-quality paradox: model, task, and even individual-instance
performance can improve when images deviate from human-perceived fidelity.
Off-the-shelf restoration pipelines fail to reconcile these idiosyncratic
preferences. To close the gap, we introduce Visual-Quality Test-Time Tuning
(VQ-TTT)-a lightweight adaptation module that: (1) inserts a learnable,
low-rank kernel before the frozen vision encoder to modulate frequency content;
and (2) fine-tunes only shallow vision-encoder layers via LoRA. VQ-TTT
dynamically adjusts each input image in a single forward pass, aligning it with
task-specific model preferences. Across the evaluated MLLMs and all datasets,
VQ-TTT lifts significant average accuracy, with no external models, cached
features, or extra training data. These findings redefine ``better'' visual
inputs for MLLMs and highlight the need for adaptive, rather than universally
``clean'', imagery, in the new era of AI being the main data customer.
