Automatic Facial Expression Recognition Method Using Deep Convolutional Neural Network

Erfani, Seyedeh H.

doi:10.22044/jadm.2020.8801.2018

Document Type : Original/Review Paper

Author

Seyedeh H. Erfani

School of Engineering, Damghan University, Damghan, Iran.

https://doi.org/10.22044/jadm.2020.8801.2018

Abstract

Facial expressions are part of human language and are often used to convey emotions. Since humans are very different in their emotional representation through various media, the recognition of facial expression becomes a challenging problem in machine learning methods. Emotion and sentiment analysis also have become new trends in social media. Deep Convolutional Neural Network (DCNN) is one of the newest learning methods in recent years that model a human's brain. DCNN achieves better accuracy with big data such as images. In this paper an automatic facial expression recognition (FER) method using the deep convolutional neural network is proposed. In this work, a way is provided to overcome the overfitting problem in training the deep convolutional neural network for FER, and also an effective pre-processing phase is proposed that is improved the accuracy of facial expression recognition. Here the results for recognition of seven emotional states (neutral, happiness, sadness, surprise, anger, fear, disgust) have been presented by applying the proposed method on the two largely used public datasets (JAFFE and CK+). The results show that in the proposed method, the accuracy of the FER is better than traditional FER methods and is about 98.59% and 96.89% for JAFFE and CK+ datasets, respectively.

Keywords

20.1001.1.23225211.2021.9.2.2.5

References

[1] J. Lia, D. Zhanga, J. Zhanga, T. Lia, Y. Xiaa, Q. Yana, and L. Xuna, “Facial Expression Recognition with Faster R-CNN”, Procedia Computer Science, vol. 107, pp.135-140, 2017.

[2] J. Przybyło, “Automatic recognition of facial expressions in the image and analysis of their suitability for control”, doctoral dissertation, AGH University of Science and Technology, Kraków, 2008.

[3] M. Rezaei, and V. Derhami, “Improving LNMF Performance of Facial Expression Recognition via Significant Parts Extraction using Shapley Value”, Journal of AI and Data Mining, vol. 7, no. 1, pp. 17-25, 2019.

[4] L. Chen, M. Zhou, W. Su, M. Wu, J. She, and K. Hirota, “Softmax regression based deep sparse autoencoder network for facial emotion recognition in human-robot interaction”, Information Sciences, vol. 428, pp. 49-61, 2018.

[5] Q. Mao, Q. Rao, and Y. Yu, “Hierarchical bayesian theme models for multi-pose facial expression recognition”, IEEE Trans. Multimedia, vol. 16, no. 4, pp. 861–873, 2017.

[6] L. A. Teixeira, E. De Aguiar, A. De Souza, and T. Oliveira-Santos, “Facial Expression Recognition with Convolutional Neural Networks: Coping with Few Data and the Training Sample Order”, Pattern Recognition, vol. 61, 2016.

[7] L. Zhang, and D. Tjondronegoro, “Facial expression recognition using facial movement features”, IEEE Trans. Affect. Comput., vol. 2, no. 4, pp. 219–229, 2011.

[8] C. J. Lin, W. L. Chu, C. C. Wang, G. K. Chen, and I. T. Chen, “Diagnosis of ball-bearing faults using support vector machine based on the artificial fish-swarm algorithm”, Journal of Low Frequency Noise, Vibration and Active Control, pp. 1–4. 2019 doi: 10.1177/1461348419861822

[9] B. L. Jian, C. C. Wang, C. T. Hsieh, Y. P. Kuo, M. C. Houng and H. T. Yau, “Predicting spindle displacement caused by heat using the general regression neural network”, The International Journal of Advanced Manufacturing Technology, 2019 doi: 10.1007/s00170-019-04261-5

[10] C. Shi, and C-M. Pun, “3d multi-resolution wavelet convolutional neural networks for hyper-spectral image classification”, Inf. Sci., vol. 420, pp. 49–65, 2017.

[11] Y. Wang, X. Wang, and W. Liu, “Unsupervised local deep feature for image recognition”, Inf. Sci., vol. 351, pp.67–75, 2016.

[12] V. Mayya, R. M. Pai, and M. M. Pai, “Automatic Facial Expression Recognition Using DCNN”. Procedia Computer Science, vol. 93, pp. 453-461, 2016.

[13] B. K. Kim, J. Roh, S. Y. Dong and S. Y. Lee, “Hierarchical committee of deep convolutional neural networks for robust facial expression recognition”, Journal on Multimodal User Interfaces, vol. 10, no. 2, 2016.

[14] X. Wang, R. Guo, and C. Kambhamettu, “Deeply-learned feature for age estimation”, In: Applications of Computer Vision (WACV), IEEE Winter Conference on., pp. 534–541, 2015.

[15] G. Levi, and T. Hassncer, “Age and gender classification using convolutional neural networks”, In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2015 IEEE Conference on., pp. 34–42, 2015.

[16]M. K. A. E. Meguid, and M. D. Levine, “Fully automated recognition of spontaneous facial expressions in videos using random forest classifiers”, IEEE Transactions on Affective Computing, vol. 5, no. 2, pp. 141–154, 2014.

[17] C. Turan, and K. M. Lam, “Region-based feature fusion for facial expression recognition”, International Conference on Image Processing (ICIP), in: 2014 IEEE, pp. 5966–5970, 2014.

[18] P. Lucey, J. F. Chon, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The Extended Cohn-Kanade Dataset: A complete dataset for action unit and emotion-specified expression”, In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, pp. 94–101, 2010.

[19] L. Nwosu, H. Wang, L. Jiang, I. Unwala, X. Yang, and T. Zhang, “Deep Convolutional Neural Network for Facial Expression Recognition using Facial Parts”, IEEE 15th Intl Conf on Dependable, Autonomic and Secure Computing, pp. 1318–1321, 2017.

[20] P. Burkert, F. Trier, M. Z. Afzal, A. Dengel, and M. Liwicki, “Dexpression: Deep convolutional neural network for expression recognition”, CoRR abs/1509.05371, 2015.

[21]P. Liu, S. Li, S. Shan, and X. Chen, “Facial expression recognition via a boosted deep belief network”, in: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1805–1812, 2014.

[22] D. K. Jain, P. Shamsolmoalib, and P. Sehdev, “Extended Deep Neural Network for Facial Emotion Recognition”, Extended Deep Neural Network for Facial Emotion Recognition, 2019.

[23] H. C. Santiago, T. Ren, and G. D. C. Cavalcanti, “Facial expression Recognition based on Motion Estimation”, International Joint Conference Neural Networks (IJCNN), 2016.

[24] D. Hamester, P. Barros and S. Wermter, “Face Expression Recognition with a 2-Channel Convolutional Neural Network”, International Joint Conference on Neural Networks (IJCNN), 2015.

[25] Z. Qawaqneh, A. Abu Mallouh, and B. D. Barkana, “Deep Convolutional Neural Network for Age Estimation based on VGG-Face Model”, CoRR abs/1709.01664, 2017.

[26] P. Viola, and M. Jones, “Rapid object detection using a boosted cascade of simple features”. In Proc. of CVPR.2001.

[27] A. Krizhevsky, I. Sutskever and G. E. Hinton, “Imagenet classification with deep convolutional neural networks”, In Advances in neural information processing systems, pp. 1097–1105, 2012.

[28] M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, “Coding facial expressions with gabor wavelets”, In: Proceedings of the 3rd. International Conference on Face and Gesture Recognition; FG ’98. Washington, DC, USA: IEEE Computer Society, 1998.

Automatic Facial Expression Recognition Method Using Deep Convolutional Neural Network

References

References

Volume 9, Issue 2April 2021Pages 153-159

Volume 9, Issue 2
April 2021
Pages 153-159