Real-time robust recognition of speakers' emotions and characteristics on mobile platforms

Florian Eyben, Bernd Huber, Erik Marchi, Dagmar Schuller, Bjorn Schuller

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

10 Scopus citations

Abstract

We demonstrate audEERING's sensAI technology running natively on low-resource mobile devices applied to emotion analytics and speaker characterisation tasks. A showcase application for the Android platform is provided, where au-dEERING's highly noise robust voice activity detection based on Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) is combined with our core emotion recognition and speaker characterisation engine natively on the mobile device. This eliminates the need for network connectivity and allows to perform robust speaker state and trait recognition efficiently in real-time without network transmission lags. Real-time factors are benchmarked for a popular mobile device to demonstrate the efficiency, and average response times are compared to a server based approach. The output of the emotion analysis is visualized graphically in the arousal and valence space alongside the emotion category and further speaker characteristics.

Original languageEnglish
Title of host publication2015 International Conference on Affective Computing and Intelligent Interaction, ACII 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages778-780
Number of pages3
ISBN (Electronic)9781479999538
DOIs
StatePublished - 2 Dec 2015
Externally publishedYes
Event2015 International Conference on Affective Computing and Intelligent Interaction, ACII 2015 - Xi'an, China
Duration: 21 Sep 201524 Sep 2015

Publication series

Name2015 International Conference on Affective Computing and Intelligent Interaction, ACII 2015

Conference

Conference2015 International Conference on Affective Computing and Intelligent Interaction, ACII 2015
Country/TerritoryChina
CityXi'an
Period21/09/1524/09/15

Keywords

  • Affect
  • Android
  • Emotion
  • Mobile computing
  • Paralinguistics
  • openSMILE

Fingerprint

Dive into the research topics of 'Real-time robust recognition of speakers' emotions and characteristics on mobile platforms'. Together they form a unique fingerprint.

Cite this