MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
Journal:
arXiv
Published Date:
Jul 10, 2025
Abstract
Vector Quantized Variational Autoencoders (VQ-VAEs) are fundamental models
that compress continuous visual data into discrete tokens. Existing methods
have tried to improve the quantization strategy for better reconstruction
quality, however, there still exists a large gap between VQ-VAEs and VAEs. To
narrow this gap, we propose \NickName, a novel method to augment the
representation capability of discrete codebooks, facilitating easier
optimization for codebooks and minimizing information loss, thereby enhancing
reconstruction quality. Specifically, we propose to retain the latent dimension
to preserve encoded features and incorporate a set of sub-codebooks for
quantization. Furthermore, we construct comprehensive zero-shot benchmarks
featuring resolutions of 512p and 2k to evaluate the reconstruction performance
of existing methods rigorously. \NickName~achieves the \textbf{state-of-the-art
performance on both ImageNet and $8$ zero-shot benchmarks} across all VQ-VAEs.
Notably, compared with SD-VAE, we outperform them on ImageNet significantly,
with rFID $\textbf{0.49}$ v.s. $\textbf{0.91}$, and achieve superior PSNR on
all zero-shot benchmarks. These results highlight the superiority of
\NickName~in reconstruction and pave the way for preserving fidelity in HD
image processing tasks. Code will be publicly available at
https://github.com/MKJia/MGVQ.
