Bridging Video Quality Scoring and Justification via Large Multimodal Models
Journal:
arXiv
Published Date:
Jun 26, 2025
Abstract
Classical video quality assessment (VQA) methods generate a numerical score
to judge a video's perceived visual fidelity and clarity. Yet, a score fails to
describe the video's complex quality dimensions, restricting its applicability.
Benefiting from the linguistic output, adapting video large multimodal models
(LMMs) to VQA via instruction tuning has the potential to address this issue.
The core of the approach lies in the video quality-centric instruction data.
Previous explorations mainly focus on the image domain, and their data
generation processes heavily rely on human quality annotations and proprietary
systems, limiting data scalability and effectiveness. To address these
challenges, we propose the Score-based Instruction Generation (SIG) pipeline.
Specifically, SIG first scores multiple quality dimensions of an unlabeled
video and maps scores to text-defined levels. It then explicitly incorporates a
hierarchical Chain-of-Thought (CoT) to model the correlation between specific
dimensions and overall quality, mimicking the human visual system's reasoning
process. The automated pipeline eliminates the reliance on expert-written
quality descriptions and proprietary systems, ensuring data scalability and
generation efficiency. To this end, the resulting Score2Instruct (S2I) dataset
contains over 320K diverse instruction-response pairs, laying the basis for
instruction tuning. Moreover, to advance video LMMs' quality scoring and
justification abilities simultaneously, we devise a progressive tuning strategy
to fully unleash the power of S2I. Built upon SIG, we further curate a
benchmark termed S2I-Bench with 400 open-ended questions to better evaluate the
quality justification capacity of video LMMs. Experimental results on the
S2I-Bench and existing benchmarks indicate that our method consistently
improves quality scoring and justification capabilities across multiple video
LMMs.
