Comparison of the Accuracy of The Bahasa Isyarat Indonesia (BISINDO) Detection System Using CNN and RNN Algorithm for Implementation on Android

I Gusti Agung Oka Aryananda; Febriliyan Samopa

doi:10.57152/malcom.v4i3.1465

Authors

I Gusti Agung Oka Aryananda Institut Teknologi Sepuluh Nopember
Febriliyan Samopa Institut Teknologi Sepuluh Nopember

DOI:

https://doi.org/10.57152/malcom.v4i3.1465

Keywords:

Sign Language, Bahasa Isyarat Indonesia (BISINDO), Convolutional Neural Netwroks (CNN), Recurrent Neural Nerwork (RNN), Image Processing

Abstract

Communication is a process of exchanging information that aims to establish relationships between humans. Communication difficulties are an obstacle for people with deaf disabilities or often called Deaf Friends, where they find it difficult to interact with friends around them. Sign language is the main medium of communication used worldwide by people with disabilities i.e. deaf and speech impaired. Communication between deaf people and those around them is often an obstacle because most people do not understand sign language which is often used as a medium of communication by deaf people. In dealing with this problem, researchers want to analyze the accuracy level of the Android-Based Bahasa Isyarat Indonesia Detection System (BISINDO) using the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) methods in order to determine which methods can be implemented to an Android device. This study shows that Convolutional Neural Network (CNN) has a greater and more stable accuracy rate compared to the Recurrent Neural Network (RNN) model where the CNN model produces an accuracy rate of 89%, and indicates that the ability to recognize images based on the division of Bahasa Isyarat Indonesia (BISINDO) alphabetic classes is good..

Downloads

Download data is not yet available.

References

F. A. A. Kuen, “Peranan Komunikasi Antarpribadi Terhadap Hubungan Masyarakat Ikecamatan Tamalate Kelurahan Mangasa Kota Makassar,” J. Ilm. Pranata Edu, vol. 1, no. 1, pp. 39–47, 2019, doi: 10.36090/jipe.v1i1.186.

L. Arisandi and B. Satya, “Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network,” J. Sist. Cerdas, vol. 5, no. 3, pp. 135–146, 2022, doi: 10.37396/jsc.v5i3.262.

A. Sani and S. Rahmadinni, “Deteksi Gestur Tangan Berbasis Pengolahan Citra,” J. Rekayasa Elektr., vol. 18, no. 2, pp. 115–124, 2022, doi: 10.17529/jre.v18i2.25147.

E. Sihabudin, R. K. Niswatin, and L. S. Wahyuniar, Penerjemah Bahasa Isyarat Menggunakan Tensorflow Skripsi. Kediri: Universitas Nusantara PGRI, 2022. [Online]. Available: http://repository.unpkediri.ac.id/id/eprint/7991

M. Zhillan, A. Rashif, S. Aulia, and S. Hadiyoso, “Penerjemah Huruf Vokal Pada Bahasa Isyarat Indonesia (Bisindo) Menjadi Audio Berbasis Image Processing Translator of Vowels At Bahasa Isyarat Indonesia (Bisindo) Into Image Processing-Based Audio,” vol. 9, no. 3, pp. 1087–1095, 2023.

Z. Kat?lm?? and C. Karakuzu, “Double handed dynamic Turkish Sign Language recognition using Leap Motion with meta learning approach,” Expert Syst. Appl., vol. 228, no. May, 2023, doi: 10.1016/j.eswa.2023.120453.

Lu W.Tong Z.Chu J., “Dynamic hand gesture recognition with leap motion controller,” IEEE Signal Process. Lett., vol. 23, no. 9, pp. 1188–1192, 2016, doi: 10.1109/LSP.2016.2590470.

Kersen and W. Widhiarso, “Penerapan Metode Convolutional Neural Network Dalam Klasifikasi Bahasa Isyarat,” MDP STUDENT Conf., vol. 2, no. 1, pp. 244–249, 2023, doi: https://doi.org/10.35957/mdp-sc.v2i1.4221.

D. Yolanda, K. Gunadi, and E. Setyati, “Pengenalan Alfabet Bahasa Isyarat Tangan Secara Real-Time dengan Menggunakan Metode Convolutional Neural Network dan Recurrent Neural Network,” J. Infra, vol. 8, no. 1, pp. 203–208, 2020, [Online]. Available: https://publication.petra.ac.id/index.php/teknik-informatika/article/view/9791

R. Andrian, “Sistem Informasi Tunjangan Kinerja Untuk Menentukan Tambahan Penghasilan Pegawai Negeri Sipil Menggunakan Metode Design Science Research,” JTIM J. Teknol. Inf. dan Multimed., vol. 2, no. 1, pp. 1–11, 2020, doi: 10.35746/jtim.v2i1.78.

B. Gledson, K. Rogage, A. Thompson, and H. Ponton, “Reporting on the Development of a Web-Based Prototype Dashboard for Construction Design Managers , Achieved through Design Science Research Methodology ( DSRM ),” 2024.

S. Siddique, S. Islam, E. E. Neon, T. Sabbir, I. T. Naheen, and R. Khan, “Deep Learning-based Bangla Sign Language Detection with an Edge Device,” Intell. Syst. with Appl., vol. 18, no. March, p. 200224, 2023, doi: 10.1016/j.iswa.2023.200224.

O. D. Nurhayati, D. Eridani, and M. H. Tsalavin, “Sistem Isyarat Bahasa Indonesia (SIBI) Metode Convolutional Neural Network Sequential secara Real Time,” J. Teknol. Inf. dan Ilmu Komput., vol. 9, no. 4, p. 819, 2022, doi: 10.25126/jtiik.2022944787.

R. H. Alfikri, M. S. Utomo, H. Februariyanti, and E. Nurwahyudi, “Pembangunan Aplikasi Penerjemah Bahasa Isyarat Dengan Metode Cnn Berbasis Android,” J. Teknoinfo, vol. 16, no. 2, p. 183, 2022, doi: 10.33365/jti.v16i2.1752.

M. Sholawati, K. Auliasari, and F. Ariwibisono, “Pengembangan Aplikasi Pengenalan Bahasa Isyarat Abjad Sibi Menggunakan Metode Convolutional Neural Network (Cnn),” JATI (Jurnal Mhs. Tek. Inform., vol. 6, no. 1, pp. 134–144, 2022, doi: 10.36040/jati.v6i1.4507.

M. B. S. Bakti and Y. M. Pranoto, “Pengenalan Angka Sistem Isyarat Bahasa Indonesia Dengan Menggunakan Metode Convolutional Neural Network,” Semin. Nas. Inov. Teknol., pp. 11–16, 2019.

J. M. Han, Y. Q. Ang, A. Malkawi, and H. W. Samuelson, “Using recurrent neural networks for localized weather prediction with combined use of public airport data and on-site measurements,” Build. Environ., vol. 192, 2021, doi: 10.1016/j.buildenv.2021.107601.

S. Masood, A. Srivastava, H. C. Thuwal, and M. Ahmad, Real-Time Sign Language Gesture ( Word ) Recognition from Video Sequences Using CNN and RNN. Springer Singapore. doi: 10.1007/978-981-10-7566-7.

Y. Bengio, P. Simard, P. Frasconi, and S. Member, “Learning Long-Term Dependencies with Gradient Descent is Difficult,” vol. 5, no. 2, 1994.

D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, pp. 1–13, 2020, doi: 10.1186/s12864-019-6413-7.

A. NOER, “Bahasa Isyarat Indonesia (BISINDO) Alphabets,” Kaggle, 2022. https://www.kaggle.com/datasets/achmadnoer/alfabet-bisindo