Locality-aware Parallel Decoding for Efficient Autoregressive Image Generation
Journal:
arXiv
Published Date:
Jul 2, 2025
Abstract
We present Locality-aware Parallel Decoding (LPD) to accelerate
autoregressive image generation. Traditional autoregressive image generation
relies on next-patch prediction, a memory-bound process that leads to high
latency. Existing works have tried to parallelize next-patch prediction by
shifting to multi-patch prediction to accelerate the process, but only achieved
limited parallelization. To achieve high parallelization while maintaining
generation quality, we introduce two key techniques: (1) Flexible Parallelized
Autoregressive Modeling, a novel architecture that enables arbitrary generation
ordering and degrees of parallelization. It uses learnable position query
tokens to guide generation at target positions while ensuring mutual visibility
among concurrently generated tokens for consistent parallel decoding. (2)
Locality-aware Generation Ordering, a novel schedule that forms groups to
minimize intra-group dependencies and maximize contextual support, enhancing
generation quality. With these designs, we reduce the generation steps from 256
to 20 (256$\times$256 res.) and 1024 to 48 (512$\times$512 res.) without
compromising quality on the ImageNet class-conditional generation, and
achieving at least 3.4$\times$ lower latency than previous parallelized
autoregressive models.
