Computational Assessment of Interest in Speech—Facing the Real-Life Challenge

Martin Wöllmer, Felix Weninger, Florian Eyben, Björn Schuller

Research output: Contribution to journalArticlepeer-review


Automatic detection of a speaker’s level of interest is of high relevance for many applications, such as automatic customer care, tutoring systems, or affective agents. However, as the latest Interspeech 2010 Paralinguistic Challenge has shown, reliable estimation of non-prototypical natural interest in spontaneous conversations independent of the subject still remains a challenge. In this article, we introduce a fully automatic combination of brute-forced acoustic features, linguistic analysis, and non-linguistic vocalizations, exploiting cross-entity information in an early feature fusion. Linguistic information is based on speech recognition by a multi-stream approach fusing context-sensitive phoneme predictions and standard acoustic features. We provide subject-independent results for interest assessment using Bidirectional Long Short-Term Memory networks on the official Challenge task and show that our proposed system leads to the best recognition accuracies that have ever been reported for this task. The according TUM AVIC corpus consists of highly spontaneous speech from face-to-face commercial presentations. The techniques presented in this article are also used in the SEMAINE system, which features an emotion sensitive embodied conversational agent.

Original languageEnglish
Article number225
Pages (from-to)225-234
Number of pages10
JournalKI - Kunstliche Intelligenz
Issue number3
StatePublished - 1 Aug 2011


  • Affective computing
  • Interest recognition
  • Long short-term memory
  • Recurrent neural networks


Dive into the research topics of 'Computational Assessment of Interest in Speech—Facing the Real-Life Challenge'. Together they form a unique fingerprint.

Cite this