Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior
Journal:
arXiv
Published Date:
Jan 31, 2025
Abstract
Recent advancements in diffusion models have been leveraged to address
inverse problems without additional training, and Diffusion Posterior Sampling
(DPS) (Chung et al., 2022a) is among the most popular approaches. Previous
analyses suggest that DPS accomplishes posterior sampling by approximating the
conditional score. While in this paper, we demonstrate that the conditional
score approximation employed by DPS is not as effective as previously assumed,
but rather aligns more closely with the principle of maximizing a posterior
(MAP). This assertion is substantiated through an examination of DPS on 512x512
ImageNet images, revealing that: 1) DPS's conditional score estimation
significantly diverges from the score of a well-trained conditional diffusion
model and is even inferior to the unconditional score; 2) The mean of DPS's
conditional score estimation deviates significantly from zero, rendering it an
invalid score estimation; 3) DPS generates high-quality samples with
significantly lower diversity. In light of the above findings, we posit that
DPS more closely resembles MAP than a conditional score estimator, and
accordingly propose the following enhancements to DPS: 1) we explicitly
maximize the posterior through multi-step gradient ascent and projection; 2) we
utilize a light-weighted conditional score estimator trained with only 100
images and 8 GPU hours. Extensive experimental results indicate that these
proposed improvements significantly enhance DPS's performance. The source code
for these improvements is provided in
https://github.com/tongdaxu/Rethinking-Diffusion-Posterior-Sampling-From-Conditional-Score-Estimator-to-Maximizing-a-Posterior.
