Towards real-time stereo using non-uniform image sampling and sparse dynamic programming

Constructing the 3D mesh of a scene from stereo images is a major task in computer vision. It usually involves several steps including stereo matching and meshing. Unfortunately, the time required to generate the 3D mesh is time demanding due to the large amount of pixels to be processed. In this work, we propose a framework to accelerate the overall process. The key issue is to first reduce the number of pixels by approximating an image with a content adaptive mesh. The nodes of the mesh are sparse and they represent the non-uniform samples of the image. To benefit from the reduced set of pixels, we formulate a dynamic programming based stereo matching algorithm which computes the depth only at the sparse samples. We then show by setting up some tests using some real images that the non-uniform samples are sufficient to recover the original dense depth map of the scene by interpolating them using the mesh. The results obtained also show that the employment of the proposed strategy reduces the overall processing time of stereo matching to more than 50% of the original time. We are now able to construct scenes in real-time using less computational resources.