Tracking of facial feature points by combining singular tracking results with a 3D Active Shape Model

Accurate 3D tracking of facial feature points from one monocular video sequence is appealing for many applications in human-machine interaction. In this work facial feature points are tracked with a Kanade-Lucas-Tomasi (KLT) feature tracker and the tracking results are linked with a 3D Active Shape Model (ASM). Thus, the efficient Gauss-Newton method is not solving for the shift of each facial feature point separately but for the 3D position, rotation and the 3D ASM parameters which are the same for all feature points. Thereby, not only the facial feature points are tracked more robustly but also the 3D position and the 3D ASM parameters can be extracted. The Jacobian matrix for the Gauss-Newton optimization is split via chain rule and the computations per frame are further reduced. The algorithm is evaluated on the basis of three handlabeled video sequences and it outperforms the KLT feature tracker. The results are also comparable to two other tracking algorithms presented recently, whereas the method proposed in this work is computationally less intensive.