TY - GEN
T1 - You Sound Like Your Counterpart
T2 - 20th International Conference on Speech and Computer, SPECOM 2018
AU - Han, Jing
AU - Schmitt, Maximilian
AU - Schuller, Björn
N1 - Publisher Copyright:
© 2018, Springer Nature Switzerland AG.
PY - 2018
Y1 - 2018
N2 - In social interaction, people tend to mimic their conversational partners both when they agree and disagree. Research on this phenomenon is complex but not recent in theory, and related studies show that mimicry can enhance social relationships, increase affiliation and rapport. However, automatically recognising such a phenomenon is still in its early development. In this paper, we analyse mimicry in the speech domain and propose a novel method by using hand-crafted low-level acoustic descriptors and autoencoders (AEs). Specifically, for each conversation, two AEs are built, one for each speaker. After training, the acoustic features of one speaker are tested with the AE that is trained on the features of her counterpart. The proposed approach is evaluated on a database consisting of almost 400 subjects from 6 different cultures, recorded in-the-wild. By calculating the AE’s reconstruction errors of all speakers and analysing the errors at different times in their interactions, we show that, albeit to different degrees from culture to culture, mimicry arises in most interactions.
AB - In social interaction, people tend to mimic their conversational partners both when they agree and disagree. Research on this phenomenon is complex but not recent in theory, and related studies show that mimicry can enhance social relationships, increase affiliation and rapport. However, automatically recognising such a phenomenon is still in its early development. In this paper, we analyse mimicry in the speech domain and propose a novel method by using hand-crafted low-level acoustic descriptors and autoencoders (AEs). Specifically, for each conversation, two AEs are built, one for each speaker. After training, the acoustic features of one speaker are tested with the AE that is trained on the features of her counterpart. The proposed approach is evaluated on a database consisting of almost 400 subjects from 6 different cultures, recorded in-the-wild. By calculating the AE’s reconstruction errors of all speakers and analysing the errors at different times in their interactions, we show that, albeit to different degrees from culture to culture, mimicry arises in most interactions.
KW - Affective computing
KW - Computational paralinguistics
KW - Conversation analysis
UR - http://www.scopus.com/inward/record.url?scp=85053788466&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-99579-3_20
DO - 10.1007/978-3-319-99579-3_20
M3 - Conference contribution
AN - SCOPUS:85053788466
SN - 9783319995786
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 188
EP - 197
BT - Speech and Computer - 20th International Conference, SPECOM 2018, Proceedings
A2 - Potapova, Rodmonga
A2 - Jokisch, Oliver
A2 - Karpov, Alexey
PB - Springer Verlag
Y2 - 18 September 2018 through 22 September 2018
ER -