Affective Image Content Analysis: Two Decades Review and New Perspectives

Sicheng Zhao, Xingxu Yao, Jufeng Yang, Guoli Jia, Guiguang Ding, Tat Seng Chua, Bjorn W. Schuller, Kurt Keutzer

Research output: Contribution to journalReview articlepeer-review

85 Scopus citations

Abstract

Images can convey rich semantics and induce various emotions in viewers. Recently, with the rapid advancement of emotional intelligence and the explosive growth of visual data, extensive research efforts have been dedicated to affective image content analysis (AICA). In this survey, we will comprehensively review the development of AICA in the recent two decades, especially focusing on the state-of-the-art methods with respect to three main challenges - the affective gap, perception subjectivity, and label noise and absence. We begin with an introduction to the key emotion representation models that have been widely employed in AICA and description of available datasets for performing evaluation with quantitative comparison of label noise and dataset bias. We then summarize and compare the representative approaches on (1) emotion feature extraction, including both handcrafted and deep features, (2) learning methods on dominant emotion recognition, personalized emotion prediction, emotion distribution learning, and learning from noisy data or few labels, and (3) AICA based applications. Finally, we discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.

Original languageEnglish
Pages (from-to)6729-6751
Number of pages23
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume44
Issue number10
DOIs
StatePublished - 1 Oct 2022
Externally publishedYes

Keywords

  • Affective computing
  • emotion feature extraction
  • emotional intelligence
  • image emotion
  • machine learning

Fingerprint

Dive into the research topics of 'Affective Image Content Analysis: Two Decades Review and New Perspectives'. Together they form a unique fingerprint.

Cite this