New Avenues in Audio Intelligence: Towards Holistic Real-life Audio Understanding

Björn Schuller, Alice Baird, Alexander Gebhard, Shahin Amiriparian, Gil Keren, Maximilian Schmitt, Nicholas Cummins

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


Computer audition (i.e., intelligent audio) has made great strides in recent years; however, it is still far from achieving holistic hearing abilities, which more appropriately mimic human-like understanding. Within an audio scene, a human listener is quickly able to interpret layers of sound at a single time-point, with each layer varying in characteristics such as location, state, and trait. Currently, integrated machine listening approaches, on the other hand, will mainly recognise only single events. In this context, this contribution aims to provide key insights and approaches, which can be applied in computer audition to achieve the goal of a more holistic intelligent understanding system, as well as identifying challenges in reaching this goal. We firstly summarise the state-of-the-art in traditional signal-processing-based audio pre-processing and feature representation, as well as automated learning such as by deep neural networks. This concerns, in particular, audio interpretation, decomposition, understanding, as well as ontologisation. We then present an agent-based approach for integrating these concepts as a holistic audio understanding system. Based on this, concluding, avenues are given towards reaching the ambitious goal of ‘holistic human-parity’ machine listening abilities.

Original languageEnglish
JournalTrends in Hearing
StatePublished - 2021
Externally publishedYes


  • audio intelligence
  • computer audition
  • machine learning


Dive into the research topics of 'New Avenues in Audio Intelligence: Towards Holistic Real-life Audio Understanding'. Together they form a unique fingerprint.

Cite this