Emotion Recognition in Naturalistic Speech and Language-A Survey

This chapter provides an overview over recent developments in naturalistic emotion recognition based on acoustic and linguistic cues. It discusses a variety of use-cases where emotion recognition can improve quality of service and quality of life. The chapter describes the existing corpora of emotional speech data relating to such scenarios, the underlying theory of emotion modeling, and the need for an optimal unit of analysis. It focuses on the challenges for real-life applications that have become evident: non-prototypicality; lack of solid ground truth and data sparsity; generalization across application scenarios, languages, and cultures; requirements of real-time and incremental processing; robustness with respect to acoustic conditions; and appropriate evaluation measures that reflect real-life settings. The chapter concludes by giving further directions for the field, including novel strategies to augment training data by synthesis and (semi-)unsupervised learning, as well as joint learning of other paralinguistic features by mutual information exploitation.