Concept Unlearning by Modeling Key Steps of Diffusion Process
Journal:
arXiv
Published Date:
Jul 9, 2025
Abstract
Text-to-image diffusion models (T2I DMs), represented by Stable Diffusion,
which generate highly realistic images based on textual input, have been widely
used. However, their misuse poses serious security risks. While existing
concept unlearning methods aim to mitigate these risks, they struggle to
balance unlearning effectiveness with generative retainability.To overcome this
limitation, we innovatively propose the Key Step Concept Unlearning (KSCU)
method, which ingeniously capitalizes on the unique stepwise sampling
characteristic inherent in diffusion models during the image generation
process. Unlike conventional approaches that treat all denoising steps equally,
KSCU strategically focuses on pivotal steps with the most influence over the
final outcome by dividing key steps for different concept unlearning tasks and
fine-tuning the model only at those steps. This targeted approach reduces the
number of parameter updates needed for effective unlearning, while maximizing
the retention of the model's generative capabilities.Through extensive
benchmark experiments, we demonstrate that KSCU effectively prevents T2I DMs
from generating undesirable images while better retaining the model's
generative capabilities. Our code will be released.
