Taming the Tri-Space Tension: ARC-Guided Hallucination Modeling and Control for Text-to-Image Generation
Journal:
arXiv
Published Date:
Jul 7, 2025
Abstract
Despite remarkable progress in image quality and prompt fidelity,
text-to-image (T2I) diffusion models continue to exhibit persistent
"hallucinations", where generated content subtly or significantly diverges from
the intended prompt semantics. While often regarded as unpredictable artifacts,
we argue that these failures reflect deeper, structured misalignments within
the generative process. In this work, we propose a cognitively inspired
perspective that reinterprets hallucinations as trajectory drift within a
latent alignment space. Empirical observations reveal that generation unfolds
within a multiaxial cognitive tension field, where the model must continuously
negotiate competing demands across three key critical axes: semantic coherence,
structural alignment, and knowledge grounding. We then formalize this
three-axis space as the \textbf{Hallucination Tri-Space} and introduce the
Alignment Risk Code (ARC): a dynamic vector representation that quantifies
real-time alignment tension during generation. The magnitude of ARC captures
overall misalignment, its direction identifies the dominant failure axis, and
its imbalance reflects tension asymmetry. Based on this formulation, we develop
the TensionModulator (TM-ARC): a lightweight controller that operates entirely
in latent space. TM-ARC monitors ARC signals and applies targeted,
axis-specific interventions during the sampling process. Extensive experiments
on standard T2I benchmarks demonstrate that our approach significantly reduces
hallucination without compromising image quality or diversity. This framework
offers a unified and interpretable approach for understanding and mitigating
generative failures in diffusion-based T2I systems.