Model CNN untuk Pengenalan Ekspresi Wajah pada Pembelajaran Pemrograman dengan Kondisi Oklusi di Kelas : A CNN Model for Recognizing Facial Expressions in Programming  Classes under Classroom Occlusions

Baharuddin Baharuddin; Muh. Wahyu Suryandi Adam; Annahl Riadi

doi:10.57152/malcom.v5i4.2322

Authors

Baharuddin Baharuddin Universitas Ichsan Sidenreng Rappang
Muh. Wahyu Suryandi Adam Universitas Ichsan Sidenreng Rappang
Annahl Riadi Universitas Ichsan Sidenreng Rappang

DOI:

https://doi.org/10.57152/malcom.v5i4.2322

Keywords:

CNN, Ekspresi, Mahasiswa, Pemrograman, Wajah

Abstract

Ekspresi wajah mahasiswa merupakan indikator non-verbal yang dapat dimanfaatkan oleh Learning Analytics (LA) untuk memantau keterlibatan dan kebingungan selama mata kuliah pemrograman (programming class). Dalam konteks riset ini, LA merujuk pada proses komputasional yang menganalisis data visual (ekspresi wajah) guna menyediakan informasi bagi pengajar dan sistem pendukung pembelajaran, baik secara waktu nyata maupun pascakelas. Kami meneliti pengenalan ekspresi wajah pada kondisi kelas nyata yang sarat oklusi (layar monitor, tas, tangan, masker, rekan mahasiswa), menggunakan CNN EfficientNet-B3 dengan transfer learning dan fine-tuning untuk mengklasifikasi empat kategori: Jenuh, Normal, Senang, dan Bingung. Evaluasi menggunakan akurasi, presisi-makro, recall-makro, F1-makro, serta confusion matrix. Pada test set, model meraih 33,33% akurasi, 0,33 presisi-makro, 0,33 recall-makro, dan 0,32 F1-makro, menegaskan tantangan pengenalan di bawah oklusi dan keterbatasan data. Akurasi validasi (26,67%) dan akurasi latih (45,31%) menunjukkan model belum cukup belajar dan terjadi kebingungan antarkelas khususnya antara Normal, Senang, dan Bingung. Meski metrik masih moderat, studi ini menyediakan baseline yang dapat direplikasi untuk FER tanggap oklusi pada konteks pendidikan pemrograman. Arah peningkatan mencakup penambahan & penyeimbangan data, augmentasi spesifik oklusi, face alignment, serta modul atensi/penanganan oklusi ringan menuju LA yang lebih tangguh di kelas nyata.

Downloads

Download data is not yet available.

References

J. H. Berssanette and A. C. De Francisco, ‘Cognitive Load Theory in the Context of Teaching and Learning Computer Programming: A Systematic Literature Review’, IEEE Trans. Educ., vol. 65, no. 3, pp. 440–449, Aug. 2022, doi: 10.1109/TE.2021.3127215.

G. Ekambaram, T. Rakkiannan, N. P, and J. E, ‘Transfer Learning Based Deep Learning Model for the Identification of Facial Expression’, in 2025 3rd International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT), Bengaluru, India: IEEE, Feb. 2025, pp. 2030–2037. doi: 10.1109/IDCIOT64235.2025.10915054.

U. Anwar et al., ‘Privacy-Preserving Facial Emotion Classification With Visual Micro-Doppler Signatures for Hearing Aid Applications’, IEEE Trans. Instrum. Meas., vol. 74, pp. 1–10, 2025, doi: 10.1109/TIM.2025.3548782.

S. Saxena, S. Tripathi, and T. S. B. Sudarshan, ‘An intelligent facial expression recognition system with emotion intensity classification’, Cognitive Systems Research, vol. 74, pp. 39–52, Aug. 2022, doi: 10.1016/j.cogsys.2022.04.001.

M. A. Khan, D. Chattaraj, S. Tadkal, T. H. Sardar, A. Chourasia, and P. Malik, ‘A Context for Human Behavior Recognition Using Facial Expression’, in 2025 International Conference on Computational, Communication and Information Technology (ICCCIT), Indore, India: IEEE, Feb. 2025, pp. 275–280. doi: 10.1109/ICCCIT62592.2025.10928104.

Ayu Lestari Perdana, Suharni, and A. Riadi, ‘Pengenalan Ekspresi Wajah Pengunjung Deal Coffee Menggunakan Metode Convolutional Neural Network (CNN)’, j. of Embedded Systems, secur. and intell. Systems, pp. 132–139, Nov. 2023, doi: 10.59562/jessi.v4i2.1024.

N. Li et al., ‘Chinese Face Dataset for Face Recognition in an Uncontrolled Classroom Environment’, IEEE Access, vol. 11, pp. 86963–86976, 2023, doi: 10.1109/ACCESS.2023.3302919.

S. I. Yudita, T. Mantoro, and M. A. Ayu, ‘Deep Face Recognition for Imperfect Human Face Images on Social Media using the CNN Method’, in 2021 4th International Conference of Computer and Informatics Engineering (IC2IE), Depok, Indonesia: IEEE, Sep. 2021, pp. 412–417. doi: 10.1109/IC2IE53219.2021.9649317.

V. S. Bhati, N. Tiwari, and M. Chawla, ‘A Generalized Zero-Shot Deep Learning Classifier for Emotion Recognition Using Facial Expression Images’, IEEE Access, vol. 13, pp. 18687–18700, 2025, doi: 10.1109/ACCESS.2025.3533580.

A. Gangwar, V. González-Castro, E. Alegre, and E. Fidalgo, ‘Triple-BigGAN: Semi-supervised generative adversarial networks for image synthesis and classification on sexual facial expression recognition’, Neurocomputing, vol. 528, pp. 200–216, Apr. 2023, doi: 10.1016/j.neucom.2023.01.027.

S. F. Kak, F. M. Mustafa, and A. Varol, ‘Design and Enhancement of a CNN Model to Augment the Face Recognition Accuracy’, in 2022 3rd International Informatics and Software Engineering Conference (IISEC), Ankara, Turkey: IEEE, Dec. 2022, pp. 1–5. doi: 10.1109/IISEC56263.2022.9998236.

S. S. Rajput and K. V. Arya, ‘CNN Classifier based Low-resolution Face Recognition Algorithm’, in 2020 International Conference on Emerging Frontiers in Electrical and Electronic Technologies (ICEFEET), Patna, India: IEEE, Jul. 2020, pp. 1–4. doi: 10.1109/ICEFEET49149.2020.9187001.

G. C. Lee, Y.-C. Lee, and C.-C. Chiang, ‘Low-Resolution Face Recognition in Multi-person Indoor Environments Using Convolutional Neural Networks’, in 2021 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA: IEEE, Dec. 2021, pp. 1629–1633. doi: 10.1109/CSCI54926.2021.00313.

R. G. Sharma, H. Garg, and L. K. Singh, ‘A Framework for Face Mask Detection and Face Recognition of Masked Images Using ResNet50 CNN with Multi-Task Learning and Cross- Attention Mechanism’, in 2025 7th International Conference on Signal Processing, Computing and Control (ISPCC), SOLAN, India: IEEE, Mar. 2025, pp. 237–242. doi: 10.1109/ISPCC66872.2025.11039321.

P. C. Pedro Neto, J. R. Pinto, F. Boutros, N. Damer, A. F. Sequeira, and J. S. Cardoso, ‘Beyond Masks: On the Generalization of Masked Face Recognition Models to Occluded Face Recognition’, IEEE Access, vol. 10, pp. 86222–86233, 2022, doi: 10.1109/ACCESS.2022.3199014.

S. Malakar, W. Chiracharit, and K. Chamnongthai, ‘Masked Face Recognition With Generated Occluded Part Using Image Augmentation and CNN Maintaining Face Identity’, IEEE Access, vol. 12, pp. 126356–126375, 2024, doi: 10.1109/ACCESS.2024.3446652.

H. B. Fredj, S. Sghaier, and C. Souani, ‘An Efficient Face Recognition Method Using CNN’, in 2021 International Conference of Women in Data Science at Taif University (WiDSTaif ), Taif, Saudi Arabia: IEEE, Mar. 2021, pp. 1–5. doi: 10.1109/WiDSTaif52235.2021.9430209.

A. A. Albraikan et al., ‘Intelligent facial expression recognition and classification using optimal deep transfer learning model’, Image and Vision Computing, vol. 128, p. 104583, Dec. 2022, doi: 10.1016/j.imavis.2022.104583.

S. Karamizadeh, S. S. Chaeikar, and M. K. Najafabadi, ‘Enhancing Facial Recognition and Expression Analysis With Unified Zero-Shot and Deep Learning Techniques’, IEEE Access, vol. 13, pp. 43508–43519, 2025, doi: 10.1109/ACCESS.2025.3546061.

Richard Steven Immanuel Sihombing, Rafif Nauval Tuah Siregar, Vijay Sitorus, and Timotius Selar Sitompul, ‘Pengenalan Ekspresi Wajah Menggunakan Convolutional Neural Network (CNN)’, jcsr-politama, vol. 1, no. 6, pp. 89–97, Dec. 2023, doi: 10.55606/jcsrpolitama.v1i6.3046.