Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval
Journal:
arXiv
Published Date:
May 26, 2025
Abstract
Zero-Shot Composed Image Retrieval (ZS-CIR) aims to retrieve target images
given a compositional query, consisting of a reference image and a modifying
text-without relying on annotated training data. Existing approaches often
generate a synthetic target text using large language models (LLMs) to serve as
an intermediate anchor between the compositional query and the target image.
Models are then trained to align the compositional query with the generated
text, and separately align images with their corresponding texts using
contrastive learning. However, this reliance on intermediate text introduces
error propagation, as inaccuracies in query-to-text and text-to-image mappings
accumulate, ultimately degrading retrieval performance. To address these
problems, we propose a novel framework by employing a Multimodal Reasoning
Agent (MRA) for ZS-CIR. MRA eliminates the dependence on textual intermediaries
by directly constructing triplets, , using only unlabeled image data. By training on these synthetic
triplets, our model learns to capture the relationships between compositional
queries and candidate images directly. Extensive experiments on three standard
CIR benchmarks demonstrate the effectiveness of our approach. On the FashionIQ
dataset, our method improves Average R@10 by at least 7.5\% over existing
baselines; on CIRR, it boosts R@1 by 9.6\%; and on CIRCO, it increases mAP@5 by
9.5\%.
