TY - JOUR
T1 - Will Affective Computing Emerge From Foundation Models and General Artificial Intelligence? A First Evaluation of ChatGPT
AU - Amin, Mostafa M.
AU - Cambria, Erik
AU - Schuller, Bjorn W.
N1 - Publisher Copyright:
© 2001-2011 IEEE.
PY - 2023/3/1
Y1 - 2023/3/1
N2 - ChatGPT has shown the potential of emerging general artificial intelligence capabilities, as it has demonstrated competent performance across many natural language processing tasks. In this work, we evaluate the capabilities of ChatGPT to perform text classification on three affective computing problems, namely, big-five personality prediction, sentiment analysis, and suicide tendency detection. We utilize three baselines, a robust language model (RoBERTa-base), a legacy word model with pretrained embeddings (Word2Vec), and a simple bag-of-words (BoW) baseline. Results show that the RoBERTa model trained for a specific downstream task generally has a superior performance. On the other hand, ChatGPT provides decent results and is relatively comparable to the Word2Vec and BoW baselines. ChatGPT further shows robustness against noisy data, where the Word2Vec model achieves worse results due to noise. Results indicate that ChatGPT is a good generalist model that is capable of achieving good results across various problems without any specialized training; however, it is not as good as a specialized model for a downstream task.
AB - ChatGPT has shown the potential of emerging general artificial intelligence capabilities, as it has demonstrated competent performance across many natural language processing tasks. In this work, we evaluate the capabilities of ChatGPT to perform text classification on three affective computing problems, namely, big-five personality prediction, sentiment analysis, and suicide tendency detection. We utilize three baselines, a robust language model (RoBERTa-base), a legacy word model with pretrained embeddings (Word2Vec), and a simple bag-of-words (BoW) baseline. Results show that the RoBERTa model trained for a specific downstream task generally has a superior performance. On the other hand, ChatGPT provides decent results and is relatively comparable to the Word2Vec and BoW baselines. ChatGPT further shows robustness against noisy data, where the Word2Vec model achieves worse results due to noise. Results indicate that ChatGPT is a good generalist model that is capable of achieving good results across various problems without any specialized training; however, it is not as good as a specialized model for a downstream task.
UR - http://www.scopus.com/inward/record.url?scp=85157967700&partnerID=8YFLogxK
U2 - 10.1109/MIS.2023.3254179
DO - 10.1109/MIS.2023.3254179
M3 - Article
AN - SCOPUS:85157967700
SN - 1541-1672
VL - 38
SP - 15
EP - 23
JO - IEEE Intelligent Systems
JF - IEEE Intelligent Systems
IS - 2
ER -