DynaSurfGS: Dynamic Surface Reconstruction with Planar-based Gaussian Splatting

1Fudan University, 2State Key Lab of CAD&CG,Zhejiang University, 3Shanghai AI Laboratory

Abstract

Dynamic scene reconstruction has garnered significant attention in recent years due to its capabilities in high-quality and real-time rendering. Among various methodologies, constructing a 4D spatial-temporal representation, such as 4D-GS, has gained popularity for its high-quality rendered images. However, these methods often produce suboptimal surfaces, as the discrete 3D Gaussian point clouds fail to align with the object's surface precisely. To address this problem, we propose DynaSurfGS to achieve both photorealistic rendering and high-fidelity surface reconstruction of dynamic scenarios. Specifically, the DynaSurfGS framework first incorporates Gaussian features from 4D neural voxels with the planar-based Gaussian Splatting to facilitate precise surface reconstruction. It leverages normal regularization to enforce the smoothness of the surface of dynamic objects. It also incorporates the as-rigid-as-possible (ARAP) constraint to maintain the approximate rigidity of local neighborhoods of 3D Gaussians between timesteps and ensure that adjacent 3D Gaussians remain closely aligned throughout. Extensive experiments demonstrate that DynaSurfGS surpasses state-of-the-art methods in both high-fidelity surface reconstruction and photorealistic rendering.

Method

DynaSurfGS overview.

Overview of our method. Firstly, in the deformation field, we represent the spatial and temporal information of dynamic objects in Hex-Plane and use an MLP to estimate the 3D Gaussian deformation. Subsequently, ARAP regularization is applied to ensure the local rigidity of the dynamic object at different moments. Finally, planar-based Gaussian splatting is used to obtain the unbiased depth map and render the transformed 3D Gaussian to images.

360-degree video

We produce a 360-degree video showcasing the rendered images and meshes of the dynmamic objects reconstructed by our approach, along with comparative results from DG-Mesh and 4D-GS.

BibTeX

@article{cai2024dynasurfgs,
  title={DynaSurfGS: Dynamic Surface Reconstruction with Planar-based Gaussian Splatting},
  author={Cai, Weiwei and Ye, Weicai and Ye, Peng and He, Tong and Chen, Tao},
  journal={arXiv preprint arXiv:2408.13972},
  year={2024}
}