You Sound Like Your Counterpart: Interpersonal Speech Analysis

Jing Han, Maximilian Schmitt, Björn Schuller

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

In social interaction, people tend to mimic their conversational partners both when they agree and disagree. Research on this phenomenon is complex but not recent in theory, and related studies show that mimicry can enhance social relationships, increase affiliation and rapport. However, automatically recognising such a phenomenon is still in its early development. In this paper, we analyse mimicry in the speech domain and propose a novel method by using hand-crafted low-level acoustic descriptors and autoencoders (AEs). Specifically, for each conversation, two AEs are built, one for each speaker. After training, the acoustic features of one speaker are tested with the AE that is trained on the features of her counterpart. The proposed approach is evaluated on a database consisting of almost 400 subjects from 6 different cultures, recorded in-the-wild. By calculating the AE’s reconstruction errors of all speakers and analysing the errors at different times in their interactions, we show that, albeit to different degrees from culture to culture, mimicry arises in most interactions.

Original languageEnglish
Title of host publicationSpeech and Computer - 20th International Conference, SPECOM 2018, Proceedings
EditorsRodmonga Potapova, Oliver Jokisch, Alexey Karpov
PublisherSpringer Verlag
Pages188-197
Number of pages10
ISBN (Print)9783319995786
DOIs
StatePublished - 2018
Externally publishedYes
Event20th International Conference on Speech and Computer, SPECOM 2018 - Leipzig, Germany
Duration: 18 Sep 201822 Sep 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11096 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Speech and Computer, SPECOM 2018
Country/TerritoryGermany
CityLeipzig
Period18/09/1822/09/18

Keywords

  • Affective computing
  • Computational paralinguistics
  • Conversation analysis

Fingerprint

Dive into the research topics of 'You Sound Like Your Counterpart: Interpersonal Speech Analysis'. Together they form a unique fingerprint.

Cite this