A convex relaxation approach to space time multi-view 3D reconstruction

We propose a convex relaxation approach to space-time 3D reconstruction from multiple videos. Generalizing the works Unger et al. [16], Kolev et al. [8] to the 4D setting, we cast the problem of reconstruction over time as a binary labeling problem in a 4D space. We propose a variational formulation which combines a photo consistency based data term with a spatio-temporal total variation regularization. In particular, we propose a novel data term that is both faster to compute and better suited for wide-baseline camera setups when photo consistency measures are unreliable or missing. The proposed functional can be globally minimized using convex relaxation techniques. Numerous experiments on a variety of public ally available data sets demonstrate that we can compute detailed and temporally consistent reconstructions. In particular, the temporal regularization allows to reduce jittering of voxels over time.