You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting
Journal:
arXiv
Published Date:
Apr 12, 2025
Abstract
Recently, reconstructing scenes from a single panoramic image using advanced
3D Gaussian Splatting (3DGS) techniques has attracted growing interest.
Panoramic images offer a 360$\times$ 180 field of view (FoV), capturing the
entire scene in a single shot. However, panoramic images introduce severe
distortion, making it challenging to render 3D Gaussians into 2D distorted
equirectangular space directly. Converting equirectangular images to cubemap
projections partially alleviates this problem but introduces new challenges,
such as projection distortion and discontinuities across cube-face boundaries.
To address these limitations, we present a novel framework, named TPGS, to
bridge continuous panoramic 3D scene reconstruction with perspective Gaussian
splatting. Firstly, we introduce a Transition Plane between adjacent cube faces
to enable smoother transitions in splatting directions and mitigate
optimization ambiguity in the boundary region. Moreover, an intra-to-inter face
optimization strategy is proposed to enhance local details and restore visual
consistency across cube-face boundaries. Specifically, we optimize 3D Gaussians
within individual cube faces and then fine-tune them in the stitched panoramic
space. Additionally, we introduce a spherical sampling technique to eliminate
visible stitching seams. Extensive experiments on indoor and outdoor,
egocentric, and roaming benchmark datasets demonstrate that our approach
outperforms existing state-of-the-art methods. Code and models will be
available at https://github.com/zhijieshen-bjtu/TPGS.
