SAGE: Exploring the Boundaries of Unsafe Concept Domain with Semantic-Augment Erasing
Journal:
arXiv
Published Date:
Jun 11, 2025
Abstract
Diffusion models (DMs) have achieved significant progress in text-to-image
generation. However, the inevitable inclusion of sensitive information during
pre-training poses safety risks, such as unsafe content generation and
copyright infringement. Concept erasing finetunes weights to unlearn
undesirable concepts, and has emerged as a promising solution. However,
existing methods treat unsafe concept as a fixed word and repeatedly erase it,
trapping DMs in ``word concept abyss'', which prevents generalized
concept-related erasing. To escape this abyss, we introduce semantic-augment
erasing which transforms concept word erasure into concept domain erasure by
the cyclic self-check and self-erasure. It efficiently explores and unlearns
the boundary representation of concept domain through semantic spatial
relationships between original and training DMs, without requiring additional
preprocessed data. Meanwhile, to mitigate the retention degradation of
irrelevant concepts while erasing unsafe concepts, we further propose the
global-local collaborative retention mechanism that combines global semantic
relationship alignment with local predicted noise preservation, effectively
expanding the retentive receptive field for irrelevant concepts. We name our
method SAGE, and extensive experiments demonstrate the comprehensive
superiority of SAGE compared with other methods in the safe generation of DMs.
The code and weights will be open-sourced at
https://github.com/KevinLight831/SAGE.
