Not All Thats Rare Is Lost: Causal Paths to Rare Concept Synthesis
Journal:
arXiv
Published Date:
May 27, 2025
Abstract
Diffusion models have shown strong capabilities in high-fidelity image
generation but often falter when synthesizing rare concepts, i.e., prompts that
are infrequently observed in the training distribution. In this paper, we
introduce RAP, a principled framework that treats rare concept generation as
navigating a latent causal path: a progressive, model-aligned trajectory
through the generative space from frequent concepts to rare targets. Rather
than relying on heuristic prompt alternation, we theoretically justify that
rare prompt guidance can be approximated by semantically related frequent
prompts. We then formulate prompt switching as a dynamic process based on score
similarity, enabling adaptive stage transitions. Furthermore, we reinterpret
prompt alternation as a second-order denoising mechanism, promoting smooth
semantic progression and coherent visual synthesis. Through this causal lens,
we align input scheduling with the model's internal generative dynamics.
Experiments across diverse diffusion backbones demonstrate that RAP
consistently enhances rare concept generation, outperforming strong baselines
in both automated evaluations and human studies.
