Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
Journal:
arXiv
Published Date:
Jun 28, 2025
Abstract
Large Vision-Language Models (VLMs) have revolutionized computer vision,
enabling tasks such as image classification, captioning, and visual question
answering. However, they remain highly vulnerable to adversarial attacks,
particularly in scenarios where both visual and textual modalities can be
manipulated. In this study, we conduct a comprehensive reproducibility study of
"An Image is Worth 1000 Lies: Adversarial Transferability Across Prompts on
Vision-Language Models" validating the Cross-Prompt Attack (CroPA) and
confirming its superior cross-prompt transferability compared to existing
baselines. Beyond replication we propose several key improvements: (1) A novel
initialization strategy that significantly improves Attack Success Rate (ASR).
(2) Investigate cross-image transferability by learning universal
perturbations. (3) A novel loss function targeting vision encoder attention
mechanisms to improve generalization. Our evaluation across prominent VLMs --
including Flamingo, BLIP-2, and InstructBLIP as well as extended experiments on
LLaVA validates the original results and demonstrates that our improvements
consistently boost adversarial effectiveness. Our work reinforces the
importance of studying adversarial vulnerabilities in VLMs and provides a more
robust framework for generating transferable adversarial examples, with
significant implications for understanding the security of VLMs in real-world
applications.
