Supervised Contrastive Learning for Game-Play Frustration Detection from Speech

Frustration is a common response during game interactions, typically decreasing a user’s engagement and leading to game failure. Artificially intelligent methods capable to automatically detect a user’s level of frustration at an early stage are hence of great interest for game designers, since this would enable optimisation of a player’s experience in real-time. Nevertheless, research in this context is still in its infancy, mainly relying on the use of pre-trained models and fine-tuning tailored to a specific dataset. Furthermore, this lack in research is due to the limited data available and to the ambiguous labelling of frustration, which leads to outcomes which are not generalisable in the real-world. Meanwhile, contrastive loss has been considered instead of the traditional cross-entropy loss in a variety of machine learning applications, showing to be more robust for system stability alternative in self-supervised learning. Following this trend, we hypothesise that using a supervised contrastive loss might overcome the limitations of the cross-entropy loss yielded by the labels’ ambiguity. In fact, our experiments demonstrate that using the supervised contrastive method as a loss function, results improve for the automatic recognition (binary frustration vs no-frustration) of game-induced frustration from speech with an Unweighted Average Recall increase from 86.4 % to 89.9 %.