Emotion Recognition with Refined Labels for Deep Learning.

The traditional emotion classification framework usually fits all the features segments of the same trial to a fixed annotation. Considering the fact that emotion is a reaction to stimuli that lasts for varied periods, we argue that the indiscriminate annotation is equivalent to taking the emotional state as fixed within the whole trial, leading to a decrease of the classification accuracy. In this study, we attempt to alleviate this issue by developing a thresholding scheme, converting the continuous emotional trace into a three-class annotation temporally. The features within a trial are therefore assigned to varied emotional states, resulting in an improvement in the accuracy. A long short term memory (LSTM) networks-based emotion classification framework is implemented, to which the proposed thresholding scheme is applied. A subset of MAHNOB-HCI dataset with continuous emotional annotation is used. The EEG signal and frontal facial video are used for feature extraction. The experiment results demonstrate that the proposed scheme provides statistically significant improvement to the three-class classification accuracy of the EEG feature-based LSTM network (p-value = 0.0329).