SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration
Journal:
arXiv
Published Date:
Jan 2, 2025
Abstract
Video restoration poses non-trivial challenges in maintaining fidelity while
recovering temporally consistent details from unknown degradations in the wild.
Despite recent advances in diffusion-based restoration, these methods often
face limitations in generation capability and sampling efficiency. In this
work, we present SeedVR, a diffusion transformer designed to handle real-world
video restoration with arbitrary length and resolution. The core design of
SeedVR lies in the shifted window attention that facilitates effective
restoration on long video sequences. SeedVR further supports variable-sized
windows near the boundary of both spatial and temporal dimensions, overcoming
the resolution constraints of traditional window attention. Equipped with
contemporary practices, including causal video autoencoder, mixed image and
video training, and progressive training, SeedVR achieves highly-competitive
performance on both synthetic and real-world benchmarks, as well as
AI-generated videos. Extensive experiments demonstrate SeedVR's superiority
over existing methods for generic video restoration.
