Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model
Journal:
arXiv
Published Date:
Feb 24, 2025
Abstract
Room layout estimation from multiple-perspective images is poorly
investigated due to the complexities that emerge from multi-view geometry,
which requires muti-step solutions such as camera intrinsic and extrinsic
estimation, image matching, and triangulation. However, in 3D reconstruction,
the advancement of recent 3D foundation models such as DUSt3R has shifted the
paradigm from the traditional multi-step structure-from-motion process to an
end-to-end single-step approach. To this end, we introduce Plane-DUSt3R, a
novel method for multi-view room layout estimation leveraging the 3D foundation
model DUSt3R. Plane-DUSt3R incorporates the DUSt3R framework and fine-tunes on
a room layout dataset (Structure3D) with a modified objective to estimate
structural planes. By generating uniform and parsimonious results, Plane-DUSt3R
enables room layout estimation with only a single post-processing step and 2D
detection results. Unlike previous methods that rely on single-perspective or
panorama image, Plane-DUSt3R extends the setting to handle multiple-perspective
images. Moreover, it offers a streamlined, end-to-end solution that simplifies
the process and reduces error accumulation. Experimental results demonstrate
that Plane-DUSt3R not only outperforms state-of-the-art methods on the
synthetic dataset but also proves robust and effective on in the wild data with
different image styles such as cartoon. Our code is available at:
https://github.com/justacar/Plane-DUSt3R
