A Wide Evaluation of ChatGPT on Affective Computing Tasks

Mostafa M. Amin, Rui Mao, Erik Cambria, Bjorn W. Schuller

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

With the rise of foundation models, a new artificial intelligence paradigm has emerged, by simply using general purpose foundation models with prompting to solve problems instead of training a separate machine learning model for each problem. Such models have been shown to have emergent properties of solving problems that they were not initially trained on. The studies for the effectiveness of such models are still quite limited. In this work, we widely study the capabilities of the ChatGPT models, namely GPT-4 and GPT-3.5, on 13 affective computing problems, namely aspect extraction, aspect polarity classification, opinion extraction, sentiment analysis, sentiment intensity ranking, emotions intensity ranking, suicide tendency detection, toxicity detection, well-being assessment, engagement measurement, personality assessment, sarcasm detection, and subjectivity detection. We introduce a framework to evaluate the ChatGPT models on regression-based problems, such as intensity ranking problems, by modelling them as pairwise ranking classification. We compare ChatGPT against more traditional NLP methods, such as end-to-end recurrent neural networks and transformers. The results demonstrate the emergent abilities of the ChatGPT models on a wide range of affective computing problems, where GPT-3.5 and especially GPT-4 have shown strong performance on many problems, particularly the ones related to sentiment, emotions, or toxicity. The ChatGPT models fell short for problems with implicit signals, such as engagement measurement and subjectivity detection.

Original languageEnglish
Pages (from-to)2204-2212
Number of pages9
JournalIEEE Transactions on Affective Computing
Volume15
Issue number4
DOIs
StatePublished - 2024

Keywords

  • ChatGPT
  • GPT-4
  • affective computing
  • aspect-based sentiment analysis
  • emotions intensity ranking
  • engagement measurement
  • foundation models
  • personality assessment
  • sarcasm detection
  • sentiment analysis
  • sentiment intensity ranking
  • subjectivity detection
  • suicide tendency detection
  • toxicity detection
  • well-being assessment

Fingerprint

Dive into the research topics of 'A Wide Evaluation of ChatGPT on Affective Computing Tasks'. Together they form a unique fingerprint.

Cite this