Technical Report for ICML 2024 TiFA Workshop MLLM Attack Challenge: Suffix Injection and Projected Gradient Descent Can Easily Fool An MLLM
Journal:
arXiv
Published Date:
Dec 20, 2024
Abstract
This technical report introduces our top-ranked solution that employs two
approaches, \ie suffix injection and projected gradient descent (PGD) , to
address the TiFA workshop MLLM attack challenge. Specifically, we first append
the text from an incorrectly labeled option (pseudo-labeled) to the original
query as a suffix. Using this modified query, our second approach applies the
PGD method to add imperceptible perturbations to the image. Combining these two
techniques enables successful attacks on the LLaVA 1.5 model.
