Fully automatic audiovisual emotion recognition: Voice, words, and the face

The recognition of human emotions from spontaneous and non-prototypical real-life data is currently one of the most challenging tasks in the field of affective computing. This contribution presents our recent advances in assessing dimensional representations of emotion, such as arousal, expectation, power, and valence, in an audiovisual human-computer interaction scenario. We propose a fully automatic multimodal recognition approach based on context-sensitive modeling of audio and video features. Evaluations on the Audiovisual Sub-Challenge of the 2011 Audio/Visual Emotion Challenge show how accurately different affective dimensions can be recognized. Our experiments reveal that the proposed multimodal recognition system outperforms previously introduced techniques evaluated on the same task.