The effect of personality trait, age, and gender on the performance of automatic speech valence recognition

Individual differences have significant effects on the expression of emotions. One may express the emotions openly such that they are easily recognizable, and one may be less expressive. Consequently, an emotion recognizer system will be affected by the emotion expressions from different individuals. Knowing which human factors improve or deteriorate the performance of the emotion recognizer, we can train systems based on those factors and select one of those systems that corresponds to the detected human factor of the target person. In this paper, we investigate the effect of age, gender, and Big-Five personality traits (Openness to Experience, Conscientiousness, Extroversion, Agreeableness, and Neuroticism) on the performance of a speech emotion recognizer. We found that, age is the paramount factor followed by gender. Conscientiousness and Neuroticism also have a substantial effect. These findings are in congruent with the literature, meaning that the performance of a speech emotion recognizer is closely correlated with the emotion expressivity of the individuals whose speech are used for training the recognition models. Additionally, based on these findings, we create a set of simple rules to select an appropriate trained model for new speech samples. This model selection approach yields higher emotion recognition accuracy.