PermutoSDF: Fast Multi-View Reconstruction with
Implicit Surfaces using Permutohedral Lattices
CVPR 2023
Video
Overview of our PermutoSDF pipeline. 1) For a batch of pixels from the posed images, we sample rays inside the volume of interest. 2) For each sample we slice features from a multi-resolution permutohedral lattice. 3) The features from all lattice levels are concatenated. For the color network, we also concatenate additional features regarding normal of the SDF, view direction and learnable features from the SDF network. 4) A small MLP decodes the SDF or a view-dependent RGB color. 5) The final output is an SDF and color field which is volumetrically rendered and supervised only with RGB images.
Highly Detailed Reconstruction
PermutoSDF can recover highly detailed reconstructions given only 2D images and camera poses. The recovered SDF and color can be rendered into novel-views that account for view-dependant effects. Using our proposed network regularization, we can recover details at the level of pores and wrinkles.
4D Spatio-Temporal Surface
The permutohedral lattice scales linearly with input dimensionality instead of exponentially as in the case of hyper-cubical voxels. This allows us to easily represent 4D surfaces which evolve in time. In this example we use a 4D lattice to encode an evolving shape represented as an SDF in (x,y,z,t). We traverse the time dimension of the network and show that it smoothly interpolates between the different time-steps of the shape.
Comparison
Other works that encode shape as an SDF fail to recover high-frequency detail. We show that using the permutohedral lattice combined with our proposed regularization method allows the network to recover significantly more geometric detail.
Citation
Acknowledgements
The website template was borrowed from Michaël Gharbi.