Noise Conditional Variational Score Distillation
Journal:
arXiv
Published Date:
Jun 11, 2025
Abstract
We propose Noise Conditional Variational Score Distillation (NCVSD), a novel
method for distilling pretrained diffusion models into generative denoisers. We
achieve this by revealing that the unconditional score function implicitly
characterizes the score function of denoising posterior distributions. By
integrating this insight into the Variational Score Distillation (VSD)
framework, we enable scalable learning of generative denoisers capable of
approximating samples from the denoising posterior distribution across a wide
range of noise levels. The proposed generative denoisers exhibit desirable
properties that allow fast generation while preserve the benefit of iterative
refinement: (1) fast one-step generation through sampling from pure Gaussian
noise at high noise levels; (2) improved sample quality by scaling the
test-time compute with multi-step sampling; and (3) zero-shot probabilistic
inference for flexible and controllable sampling. We evaluate NCVSD through
extensive experiments, including class-conditional image generation and inverse
problem solving. By scaling the test-time compute, our method outperforms
teacher diffusion models and is on par with consistency models of larger sizes.
Additionally, with significantly fewer NFEs than diffusion-based methods, we
achieve record-breaking LPIPS on inverse problems.
