CheapNVS: Real-Time On-Device Narrow-Baseline Novel View Synthesis
Journal:
arXiv
Published Date:
Jan 24, 2025
Abstract
Single-view novel view synthesis (NVS) is a notorious problem due to its
ill-posed nature, and often requires large, computationally expensive
approaches to produce tangible results. In this paper, we propose CheapNVS: a
fully end-to-end approach for narrow baseline single-view NVS based on a novel,
efficient multiple encoder/decoder design trained in a multi-stage fashion.
CheapNVS first approximates the laborious 3D image warping with lightweight
learnable modules that are conditioned on the camera pose embeddings of the
target view, and then performs inpainting on the occluded regions in parallel
to achieve significant performance gains. Once trained on a subset of Open
Images dataset, CheapNVS outperforms the state-of-the-art despite being 10
times faster and consuming 6% less memory. Furthermore, CheapNVS runs
comfortably in real-time on mobile devices, reaching over 30 FPS on a Samsung
Tab 9+.
