ACCORD: Alleviating Concept Coupling through Dependence Regularization for Text-to-Image Diffusion Personalization
Journal:
arXiv
Published Date:
Mar 3, 2025
Abstract
Image personalization has garnered attention for its ability to customize
Text-to-Image generation using only a few reference images. However, a key
challenge in image personalization is the issue of conceptual coupling, where
the limited number of reference images leads the model to form unwanted
associations between the personalization target and other concepts. Current
methods attempt to tackle this issue indirectly, leading to a suboptimal
balance between text control and personalization fidelity. In this paper, we
take a direct approach to the concept coupling problem through statistical
analysis, revealing that it stems from two distinct sources of dependence
discrepancies. We therefore propose two complementary plug-and-play loss
functions: Denoising Decouple Loss and Prior Decouple loss, each designed to
minimize one type of dependence discrepancy. Extensive experiments demonstrate
that our approach achieves a superior trade-off between text control and
personalization fidelity.
