Content-based indexing of images and video using face detection and recognition methods

This paper presents an image and video indexing approach that combines face detection and face recognition methods. Images of a database or frames of a video sequence are scanned for faces by a Neural Network-based face detector. The extracted faces are then grouped into clusters by a combination of a face recognition method using pseudo two-dimensional Hidden Markov Models and a k-means clustering algorithm. Each resulting main cluster consists of the face images of one person. In a subsequent step the detected faces are labeled as one of the different people in the video sequence or the image database and the occurrence of the people can be evaluated. The results of the proposed approach on a TV broadcast news sequence are presented. It is demonstrated that the system is able to discriminate between three different newscasters and an interviewed person.