A variational framework for image segmentation combining motion estimation and shape regularization

Based on a geometric interpretation of the optic flow constraint equation, we propose a conditional probability on the spatio-temporal image gradient. We consistently derive a variational approach for the segmentation of the image domain into regions of homogeneous motion. The proposed energy functional extends the Mumford-Shah functional from gray value segmentation to motion segmentation. It depends on the spatio-temporal image gradient calculated from only two consecutive images of an image sequence. Moreover, it depends on motion vectors for a set of regions and a boundary separating these regions. In contrast to most alternative approaches, the problems of motion estimation and motion segmentation are jointly solved by minimizing a single functional. Numerical evaluation with both explicit and implicit (level set based) representations of the boundary shows the strengths and limitations of our approach.