Facial Expression Classification Using Convolutional Neural Network on the FER2013 Dataset
DOI:
https://doi.org/10.57152/ijatis.v3i2.3093Keywords:
Convolutional Neural Network, Deep Learning, FER2013, Image Classification, Facial Expression RecognitionAbstract
Facial expression classification is an important task in computer vision that enables systems to identify human emotions from facial images. This study aims to evaluate the performance of a Convolutional Neural Network (CNN) for classifying facial expressions using the FER2013 dataset. The dataset consists of grayscale facial images categorized into seven emotion classes: Angry, Disgust, Fear, Happy, Sad, Surprise, and Neutral. The images were normalized and augmented using rotation, shifting, zooming, and horizontal flipping to improve model generalization. A CNN architecture consisting of three convolutional layers with 32, 64, and 128 filters, followed by max-pooling, a fully connected layer, dropout, and a softmax output layer, was trained using the Adam optimizer for 20 epochs with a batch size of 32. Model performance was evaluated using accuracy, precision, recall, F1-score, and a confusion matrix. The model achieved a test accuracy of 60%, with weighted precision, recall, and F1-score of 60%. Among the seven classes, Happy achieved the highest F1-score of 79%, followed by Surprise at 73%, while Fear, Sad, and Disgust achieved F1-scores of 45%, 46%, and 47%, respectively. These results indicate that the CNN can effectively classify several facial expressions, particularly Happy and Surprise, while performance is lower on visually similar and less-represented classes. The contribution of this study is an evaluation and class-level analysis of a CNN-based facial expression classification approach on the FER2013 dataset, providing insights into performance differences across emotion categories and potential directions for improving classification performance.
References
I. J. Goodfellow et al., Challenges in Representation Learning: A Report on Three Machine Learning Contests, 2013.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105, 2012.
Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “DeepFace: Closing the Gap to Human-Level Performance in Face Verification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1701–1708.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
C. Pramerdorfer and M. Kampel, “Facial Expression Recognition Using Convolutional Neural Networks: State of the Art,” 2016.
Y. Khaireddin and Z. Chen, “Facial Emotion Recognition: State of the Art Performance on FER2013,” 2021.
O. C. Oguine, C. N. Nwankwo, and E. N. Onwuka, “Hybrid Facial Expression Recognition (FER2013) Model for Real-Time Emotion Classification and Prediction,” 2022.
M. Al-Shabi, W. P. Cheah, and T. Connie, “Facial Expression Recognition Using a Hybrid CNN-SIFT Aggregator,” 2016.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-Based Learning Applied to Document Recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” 2014.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going Deeper with Convolutions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1–9.
M. Pantic and L. J. M. Rothkrantz, “Automatic Analysis of Facial Expressions: The State of the Art,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424–1445, 2000.
M. Valstar, B. Jiang, M. Mehu, M. Pantic, and K. Scherer, “The First Facial Expression Recognition and Analysis Challenge,” in IEEE International Conference on Automatic Face and Gesture Recognition, 2011, pp. 921–926.
M. H. Yap, N. Ugail, and R. Zwiggelaar, “Advances in Facial Expression Recognition: A Survey of Methods, Benchmarks, Models, and Datasets,” Information, vol. 15, no. 3, 2024.
S. Minaee, M. Minaei, A. Abdolrashidi, and Y. Wang, “Deep Facial Expression Recognition: A Survey,” IEEE Transactions on Affective Computing, vol. 13, no. 3, pp. 1195–1215, 2022.
B. B. Yap et al., "A Review of Facial Emotion Recognition Approaches for Human–Robot Interaction," Robotics, vol. 12, no. 3, p. 72, 2023
M. Bie, H. Xu, Y. Gao, K. Song, and X. Che, “Swin-FER: Swin Transformer for Facial Expression Recognition,” Applied Sciences, vol. 14, no. 14, 2024.
Y. Wang et al., “A Survey on Facial Expression Recognition of Static and Dynamic Emotions,” 2024.
C. L. Kim and B.-G. Kim, “Few-shot Learning for Facial Expression Recognition: A Comprehensive Survey,” Journal of Real-Time Image Processing, 2023.
H. Li et al., “Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation,” 2024.
S. Li et al., "Attention Mechanism-Based CNN for Facial Expression Recognition," Comput. Electr. Eng., vol. 106, p. 108541, 2023.
Z. Wang et al., "EfficientNet-Based Facial Expression Recognition Using FER2013 Dataset," IEEE Access, vol. 11, pp. 50012–50024, 2023.
C. Fan et al., "Label Distribution Learning for Facial Expression Recognition with Class-Imbalance Problem," Pattern Recognit., vol. 141, p. 109607, 2023.
A. Dosovitskiy et al., "An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale," in Proc. ICLR, 2021.
T. Zhao et al., "Vision Transformer for Facial Expression Recognition," Sensors, vol. 23, no. 4, p. 2176, 2023.
P. Liu et al., "Facial Expression Recognition via Disentangled Representation Learning," IEEE Trans. Circuits Syst. Video Technol., vol. 34, no. 2, pp. 1145–1157, 2024.
Y. Chen et al., "Multimodal Facial Expression Recognition Using CNN and Temporal Attention," Neurocomputing, vol. 575, p. 127296, 2024.
