An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era

Andreas Triantafyllopoulos, Bjorn W. Schuller, Gokce Iymen, Metin Sezgin, Xiangheng He, Zijiang Yang, Panagiotis Tzirakis, Shuo Liu, Silvan Mertes, Elisabeth Andre, Ruibo Fu, Jianhua Tao

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human-computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. However, the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions - aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesized utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology that underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In this overview, we outline ongoing trends and summarize state-of-the-art approaches in an attempt to provide a broad overview of this exciting field.

Original languageEnglish
Pages (from-to)1355-1381
Number of pages27
JournalProceedings of the IEEE
Volume111
Issue number10
DOIs
StatePublished - 1 Oct 2023
Externally publishedYes

Keywords

  • Affective computing
  • deep learning
  • emotional voice conversion (EVC)
  • speech synthesis

Fingerprint

Dive into the research topics of 'An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era'. Together they form a unique fingerprint.

Cite this