A multiphase level set framework for motion segmentation

We present a novel variational approach for segmenting the image plane into a set of regions of piecewise constant motion on the basis of only two consecutive frames from an image sequence. To this end, we formulate the problem of estimating a motion field in the framework of Bayesian inference. Our model is based on a conditional probability for the spatio-temporal image gradient, given a particular velocity vector, and on a prior on the estimated motion field favoring motion boundaries of minimal length. The corresponding negative log likelihood is a functional which depends on motion vectors for a set of regions and on the boundary separating these regions. It can be considered an extension of the Mumford-Shah functional from intensity segmentation to motion segmentation. We propose an implementation of this functional by a multiphase level set framework. Minimizing the functional with respect to its dynamic variables results in an evolution equation for a vector-valued level set function and in an eigenvalue problem for the motion vectors. Compared to most alternative approaches, we jointly solve the problems of segmentation and motion estimation by minimizing a single functional. Numerical results both for simulated ground truth experiments and for real-world sequences demonstrate the capacity of our approach to segment several - possibly multiply connected - objects based on their relative motion.