Membership Inference Attacks on Sequence Models
Journal:
arXiv
Published Date:
Jun 5, 2025
Abstract
Sequence models, such as Large Language Models (LLMs) and autoregressive
image generators, have a tendency to memorize and inadvertently leak sensitive
information. While this tendency has critical legal implications, existing
tools are insufficient to audit the resulting risks. We hypothesize that those
tools' shortcomings are due to mismatched assumptions. Thus, we argue that
effectively measuring privacy leakage in sequence models requires leveraging
the correlations inherent in sequential generation. To illustrate this, we
adapt a state-of-the-art membership inference attack to explicitly model
within-sequence correlations, thereby demonstrating how a strong existing
attack can be naturally extended to suit the structure of sequence models.
Through a case study, we show that our adaptations consistently improve the
effectiveness of memorization audits without introducing additional
computational costs. Our work hence serves as an important stepping stone
toward reliable memorization audits for large sequence models.
