Test-Time Visual In-Context Tuning
Journal:
arXiv
Published Date:
Mar 27, 2025
Abstract
Visual in-context learning (VICL), as a new paradigm in computer vision,
allows the model to rapidly adapt to various tasks with only a handful of
prompts and examples. While effective, the existing VICL paradigm exhibits poor
generalizability under distribution shifts. In this work, we propose test-time
Visual In-Context Tuning (VICT), a method that can adapt VICL models on the fly
with a single test sample. Specifically, we flip the role between the task
prompts and the test sample and use a cycle consistency loss to reconstruct the
original task prompt output. Our key insight is that a model should be aware of
a new test distribution if it can successfully recover the original task
prompts. Extensive experiments on six representative vision tasks ranging from
high-level visual understanding to low-level image processing, with 15 common
corruptions, demonstrate that our VICT can improve the generalizability of VICL
to unseen new domains. In addition, we show the potential of applying VICT for
unseen tasks at test time. Code: https://github.com/Jiahao000/VICT.
