TY - GEN
T1 - The MuSe 2021 multimodal sentiment analysis challenge
T2 - 2nd Multimodal Sentiment Analysis Challenge and Workshop, MuSe 2021, held in conjunction with the ACM Multimedia 2021
AU - Stappen, Lukas
AU - Baird, Alice
AU - Christ, Lukas
AU - Schumann, Lea
AU - Sertolli, Benjamin
AU - Meßner, Eva Maria
AU - Cambria, Erik
AU - Zhao, Guoying
AU - Schuller, Björn W.
N1 - Publisher Copyright:
© 2021 ACM.
PY - 2021/10/24
Y1 - 2021/10/24
N2 - Multimodal Sentiment Analysis (MuSe) 2021 is a challenge focusing on the tasks of sentiment and emotion, as well as physiological-emotion and emotion-based stress recognition through more comprehensively integrating the audio-visual, language, and biological signal modalities. The purpose of MuSe 2021 is to bring together communities from different disciplines; mainly, the audio-visual emotion recognition community (signal-based), the sentiment analysis community (symbol-based), and the health informatics community. We present four distinct sub-challenges: MuSe-Wilder and MuSe-Stress which focus on continuous emotion (valence and arousal) prediction; MuSe-Sent, in which participants recognise five classes each for valence and arousal; and MuSe-Physio, in which the novel aspect of 'physiological-emotion' is to be predicted. For this year's challenge, we utilise the MuSe-CaR dataset focusing on user-generated reviews and introduce the Ulm-TSST dataset, which displays people in stressful depositions. This paper also provides detail on the state-of-the-art feature sets extracted from these datasets for utilisation by our baseline model, a Long Short-Term Memory-Recurrent Neural Network. For each sub-challenge, a competitive baseline for participants is set; namely, on test, we report a Concordance Correlation Coefficient (CCC) of .4616 CCC for MuSe-Wilder; .5088 CCC for MuSe-Stress, and .4908 CCC for MuSe-Physio. For MuSe-Sent an F1 score of 32.82% is obtained.
AB - Multimodal Sentiment Analysis (MuSe) 2021 is a challenge focusing on the tasks of sentiment and emotion, as well as physiological-emotion and emotion-based stress recognition through more comprehensively integrating the audio-visual, language, and biological signal modalities. The purpose of MuSe 2021 is to bring together communities from different disciplines; mainly, the audio-visual emotion recognition community (signal-based), the sentiment analysis community (symbol-based), and the health informatics community. We present four distinct sub-challenges: MuSe-Wilder and MuSe-Stress which focus on continuous emotion (valence and arousal) prediction; MuSe-Sent, in which participants recognise five classes each for valence and arousal; and MuSe-Physio, in which the novel aspect of 'physiological-emotion' is to be predicted. For this year's challenge, we utilise the MuSe-CaR dataset focusing on user-generated reviews and introduce the Ulm-TSST dataset, which displays people in stressful depositions. This paper also provides detail on the state-of-the-art feature sets extracted from these datasets for utilisation by our baseline model, a Long Short-Term Memory-Recurrent Neural Network. For each sub-challenge, a competitive baseline for participants is set; namely, on test, we report a Concordance Correlation Coefficient (CCC) of .4616 CCC for MuSe-Wilder; .5088 CCC for MuSe-Stress, and .4908 CCC for MuSe-Physio. For MuSe-Sent an F1 score of 32.82% is obtained.
KW - affective computing
KW - benchmark
KW - challenge
KW - electrodermal activity
KW - multimodal fusion
KW - multimodal sentiment analysis
KW - stress detection
UR - http://www.scopus.com/inward/record.url?scp=85112069481&partnerID=8YFLogxK
U2 - 10.1145/3475957.3484450
DO - 10.1145/3475957.3484450
M3 - Conference contribution
AN - SCOPUS:85112069481
T3 - MuSe 2021 - Proceedings of the 2nd Multimodal Sentiment Analysis Challenge, co-located with ACM MM 2021
SP - 5
EP - 14
BT - MuSe 2021 - Proceedings of the 2nd Multimodal Sentiment Analysis Challenge, co-located with ACM MM 2021
PB - Association for Computing Machinery, Inc
Y2 - 24 October 2021 through 24 October 2021
ER -