Unleashing High-Quality Image Generation in Diffusion Sampling Using Second-Order Levenberg-Marquardt-Langevin
Journal:
arXiv
Published Date:
May 30, 2025
Abstract
The diffusion models (DMs) have demonstrated the remarkable capability of
generating images via learning the noised score function of data distribution.
Current DM sampling techniques typically rely on first-order Langevin dynamics
at each noise level, with efforts concentrated on refining inter-level
denoising strategies. While leveraging additional second-order Hessian geometry
to enhance the sampling quality of Langevin is a common practice in Markov
chain Monte Carlo (MCMC), the naive attempts to utilize Hessian geometry in
high-dimensional DMs lead to quadratic-complexity computational costs,
rendering them non-scalable. In this work, we introduce a novel
Levenberg-Marquardt-Langevin (LML) method that approximates the diffusion
Hessian geometry in a training-free manner, drawing inspiration from the
celebrated Levenberg-Marquardt optimization algorithm. Our approach introduces
two key innovations: (1) A low-rank approximation of the diffusion Hessian,
leveraging the DMs' inherent structure and circumventing explicit
quadratic-complexity computations; (2) A damping mechanism to stabilize the
approximated Hessian. This LML approximated Hessian geometry enables the
diffusion sampling to execute more accurate steps and improve the image
generation quality. We further conduct a theoretical analysis to substantiate
the approximation error bound of low-rank approximation and the convergence
property of the damping mechanism. Extensive experiments across multiple
pretrained DMs validate that the LML method significantly improves image
generation quality, with negligible computational overhead.
