D$^2$-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models
Journal:
arXiv
Published Date:
Jan 14, 2025
Abstract
Diffusion models have achieved cutting-edge performance in image generation.
However, their lengthy denoising process and computationally intensive score
estimation network impede their scalability in low-latency and
resource-constrained scenarios. Post-training quantization (PTQ) compresses and
accelerates diffusion models without retraining, but it inevitably introduces
additional quantization noise, resulting in mean and variance deviations. In
this work, we propose D2-DPM, a dual denoising mechanism aimed at precisely
mitigating the adverse effects of quantization noise on the noise estimation
network. Specifically, we first unravel the impact of quantization noise on the
sampling equation into two components: the mean deviation and the variance
deviation. The mean deviation alters the drift coefficient of the sampling
equation, influencing the trajectory trend, while the variance deviation
magnifies the diffusion coefficient, impacting the convergence of the sampling
trajectory. The proposed D2-DPM is thus devised to denoise the quantization
noise at each time step, and then denoise the noisy sample through the inverse
diffusion iterations. Experimental results demonstrate that D2-DPM achieves
superior generation quality, yielding a 1.42 lower FID than the full-precision
model while achieving 3.99x compression and 11.67x bit-operation acceleration.
