Bayesian approaches to motion-based image and video segmentation

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations


We present a variational approach for segmenting the image plane into regions of piecewise parametric motion given two or more frames from an image sequence. Our model is based on a conditional probability for the spatio-temporal image gradient, given a particular velocity model, and on a geometric prior on the estimated motion field favoring motion boundaries of minimal length. We cast the problem of motion segmentation as one of Bayesian inference, we derive a cost functional which depends on parametric motion models for each of a set of domains and on the boundary separating them. The resulting functional can be interpreted as an extension of the Mumford-Shah functional from intensity segmentation to motion segmentation. In contrast to most alternative approaches, the problems of segmentation and motion estimation are jointly solved by continuous minimization of a single functional. Minimization results in an eigenvalue problem for the motion parameters and in a gradient descent evolution for the motion boundary. The evolution of the motion boundaries is implemented by a multiphase level set formulation which allows for the segmentation of an arbitrary number of multiply connected moving objects. We further extend this approach to the segmentation of space-time volumes of coherent motion from video sequences. To this end, motion boundaries are represented by a set of surfaces in space-time. An implementation by a higher-dimensional multiphase level set model allows the evolving surfaces to undergo topological changes. In contrast to an iterative segmentation of consecutive frame pairs, a constraint on the area of these surfaces leads to an additional temporal regularization of the computed motion boundaries. Numerical results demonstrate the capacity of our approach to segment objects based exclusively on their relative motion.

Original languageEnglish
Title of host publicationComplex Motion - First International Workshop, IWCM 2004 Revised Papers
PublisherSpringer Verlag
Number of pages20
ISBN (Print)9783540698647
StatePublished - 2007
Externally publishedYes
Event1st International Workshop on Complex Motion, IWCM 2004 - Guenzburg, Germany
Duration: 12 Oct 200414 Oct 2004

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3417 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference1st International Workshop on Complex Motion, IWCM 2004


Dive into the research topics of 'Bayesian approaches to motion-based image and video segmentation'. Together they form a unique fingerprint.

Cite this