MatDecompSDF: High-Fidelity 3D Shape and PBR Material Decomposition from Multi-View Images
Journal:
arXiv
Published Date:
Jul 7, 2025
Abstract
We present MatDecompSDF, a novel framework for recovering high-fidelity 3D
shapes and decomposing their physically-based material properties from
multi-view images. The core challenge of inverse rendering lies in the
ill-posed disentanglement of geometry, materials, and illumination from 2D
observations. Our method addresses this by jointly optimizing three neural
components: a neural Signed Distance Function (SDF) to represent complex
geometry, a spatially-varying neural field for predicting PBR material
parameters (albedo, roughness, metallic), and an MLP-based model for capturing
unknown environmental lighting. The key to our approach is a physically-based
differentiable rendering layer that connects these 3D properties to the input
images, allowing for end-to-end optimization. We introduce a set of carefully
designed physical priors and geometric regularizations, including a material
smoothness loss and an Eikonal loss, to effectively constrain the problem and
achieve robust decomposition. Extensive experiments on both synthetic and
real-world datasets (e.g., DTU) demonstrate that MatDecompSDF surpasses
state-of-the-art methods in geometric accuracy, material fidelity, and novel
view synthesis. Crucially, our method produces editable and relightable assets
that can be seamlessly integrated into standard graphics pipelines, validating
its practical utility for digital content creation.
