Pengembangan Sistem OCR untuk Pengenalan Teks Daftar Bahan pada Label Makanan Berbasis CRAFT dan CRNN: Development of an OCR System for Ingredient List Text Recognition on Food Labels Using CRAFT and CRNN

Rakeen Aria Alireza; Arrie Kurniawardhani

doi:10.57152/malcom.v6i4.2707

Authors

Rakeen Aria Alireza Universitas Islam Indonesia
Arrie Kurniawardhani Universitas Islam Indonesia

DOI:

https://doi.org/10.57152/malcom.v6i4.2707

Keywords:

CRAFT, CRNN, Deep Learning, Optical Character Recognition, Pengenalan Bahan Makanan

Abstract

Informasi daftar bahan pada label kemasan makanan berperan penting dalam membantu konsumen memahami kandungan produk, namun umumnya disajikan dalam bentuk teks fisik sehingga sulit diproses secara otomatis oleh sistem digital. Penelitian ini bertujuan untuk mengembangkan sistem Optical Character Recognition (OCR) berbasis deep learning untuk mendeteksi dan mengenali teks pada daftar bahan secara otomatis. Metode yang digunakan mengombinasikan model ClusteR-specific Assorted Feature selection (CRAFT) untuk deteksi area teks dan Convolutional Recurrent Neural Network (CRNN) untuk pengenalan teks. Model pengenalan kemudian di-fine-tuning menggunakan dataset khusus teks bahan makanan untuk meningkatkan kemampuan dalam mengenali karakteristik label. Dataset terdiri dari 6.081 sub-citra berbasis kata yang dibagi menjadi 4.868 data latih, 610 data validasi, dan 603 data uji. Evaluasi dilakukan menggunakan beberapa metrik serta simulasi pada lima citra eksternal untuk menguji performa pada kondisi nyata. Hasil menunjukkan bahwa fine-tuning meningkatkan performa, dengan kenaikan Word Accuracy dari 90,381% menjadi 94,693% dan Normalized Edit Distance (NED) dari 0,946 menjadi 0,987, serta penurunan Character Error Rate (CER) dari 0,066 menjadi 0,015 dan Word Error Rate (WER) dari 0,096 menjadi 0,053. Sistem mampu mengenali sebagian besar teks dengan baik, meskipun masih terdapat kesalahan kecil pada tanda baca dan karakter yang mirip

Downloads

Download data is not yet available.

References

X.-F. Wang, Z.-H. He, K. Wang, Y.-F. Wang, L. Zou, and Z.-Z. Wu, “A survey of text detection and recognition algorithms based on deep learning technology,” Neurocomputing, vol. 556, p. 126702, 2023, doi: https://doi.org/10.1016/j.neucom.2023.126702.

V. Guimarães, J. Nascimento, P. Viana, and P. Carvalho, “A Review of Recent Advances and Challenges in Grocery Label Detection and Recognition,” Applied Sciences 2023, Vol. 13, Page 2871, vol. 13, no. 5, p. 2871, Feb. 2023, doi: 10.3390/APP13052871.

M. Nagayi, A. Khan, T. Frank, R. Swart, and C. Nyirenda, “Evaluating OCR performance on food packaging labels in South Africa,” Oct. 2025, doi: 10.48550/arXiv.2510.03570.

H. Peng, J. Bayon, J. Recas, and M. Guijarro, “Efficient Expiration Date Recognition in Food Packages for Mobile Applications,” Algorithms, vol. 18, no. 5, May 2025, doi: 10.3390/A18050286.

H. Seitaj and V. Elangovan, “Information Extraction from Product Labels: A Machine Vision Approach,” Academy and Industry Research Collaboration Center (AIRCC), Jul. 2024, pp. 227–247. doi: 10.5121/csit.2024.141419.

J. Baek et al., “What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis,” Proceedings of the IEEE International Conference on Computer Vision, pp. 4714–4722, Dec. 2019, Accessed: Mar. 13, 2026. [Online]. Available: http://arxiv.org/abs/1904.01906

J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?,” 2014.

S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” 2009, doi: 10.1109/TKDE.2009.191.

A. W. Rosyadi, S. Ma’shumah, M. Q. Zaman, and M. R. Fajar, “Ingredients Identification Through Label Scanning Using PaddleOCR and ChatGPT for Information Retrieval,” Jurnal RESTI, vol. 8, no. 6, pp. 758–767, Dec. 2024, doi: 10.29207/RESTI.V8I6.6119.

T. Manuel, E. Saputra, A. Susanto, and J. Carmelita, “Implementation of Tesseract OCR and Bounding Box for Text Extraction on Food Nutrition Labels,” Building of Informatics, Technology and Science (BITS), vol. 6, no. 3, pp. 1403–1412, Dec. 2024, doi: 10.47065/BITS.V6I3.6107.

Y. Baek, B. Lee, D. Han, S. Yun, and H. Lee, “Character Region Awareness for Text Detection,” 2019. doi: 10.1109/CVPR.2019.00959.

B. Shi, X. Bai, and C. Yao, “An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 11, pp. 2298–2304, Jul. 2015, doi: 10.1109/TPAMI.2016.2646371.

“Ingredients Image from Food label.” Accessed: Apr. 06, 2026. [Online]. Available: https://www.kaggle.com/datasets/shensivam/ingredients-image-from-food-label

P. Singh, M. Sharma, A. Dey, and B. Raman, “Frozen in Time: Parameter-Efficient Time Series Transformers via Reservoir-Induced Feature Expansion and Fixed Random Dynamics,” 2025, doi: 10.48550/arXiv:2508.18130.

D. Karatzas et al., “ICDAR 2013 robust reading competition,” in Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, 2013, pp. 1484–1493. doi: 10.1109/ICDAR.2013.221.

C. Neudecker, K. Baierer, M. Gerber, C. Christian, A. Apostolos, and P. Stefan, “A survey of OCR evaluation tools and metrics,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Sep. 2021, pp. 13–18. doi: 10.1145/3476887.3476888.