PBCAT: Patch-based composite adversarial training against physically realizable attacks on object detection
Journal:
arXiv
Published Date:
Jun 30, 2025
Abstract
Object detection plays a crucial role in many security-sensitive
applications. However, several recent studies have shown that object detectors
can be easily fooled by physically realizable attacks, \eg, adversarial patches
and recent adversarial textures, which pose realistic and urgent threats.
Adversarial Training (AT) has been recognized as the most effective defense
against adversarial attacks. While AT has been extensively studied in the
$l_\infty$ attack settings on classification models, AT against physically
realizable attacks on object detectors has received limited exploration. Early
attempts are only performed to defend against adversarial patches, leaving AT
against a wider range of physically realizable attacks under-explored. In this
work, we consider defending against various physically realizable attacks with
a unified AT method. We propose PBCAT, a novel Patch-Based Composite
Adversarial Training strategy. PBCAT optimizes the model by incorporating the
combination of small-area gradient-guided adversarial patches and imperceptible
global adversarial perturbations covering the entire image. With these designs,
PBCAT has the potential to defend against not only adversarial patches but also
unseen physically realizable attacks such as adversarial textures. Extensive
experiments in multiple settings demonstrated that PBCAT significantly improved
robustness against various physically realizable attacks over state-of-the-art
defense methods. Notably, it improved the detection accuracy by 29.7\% over
previous defense methods under one recent adversarial texture attack.
