Comparison of Machine Learning Algorithms for Multilingual Health Hoax Classification on X and YouTube

Endin Bella Sefia; Eka Sahputra; Maie Istighosah

doi:10.57152/malcom.v6i4.2859

Authors

Endin Bella Sefia Telkom University
Eka Sahputra Telkom University
Maie Istighosah Telkom University

DOI:

https://doi.org/10.57152/malcom.v6i4.2859

Keywords:

Classification, Health Hoax, Machine Learning, Multilingual, Youtube

Abstract

Short health-related messages on social media can influence how users interpret diseases, vaccines, and treatment issues. Because this type of content often appears as brief posts or video titles, automatic classification is needed to help identify misleading claims. This research evaluates three machine learning algorithms, namely Naïve Bayes, Support Vector Machine, and Logistic Regression, for health-hoax classification on X and YouTube. The data were collected from January 2020 to January 2026 and consisted of Indonesian and English records. English texts were translated into Indonesian so that all records could be processed in one language pipeline. The processing stages included text cleaning, normalization, tokenization, stemming, stopword removal, IndoBERT-based pseudo-labeling, TF-IDF feature representation, Random Under Sampling, and parameter tuning using GridSearchCV. The final dataset contained 10,983 records, consisting of 7,157 non-hoax and 3,826 hoax records. The experimental results show that Support Vector Machine produced the highest weighted F1-score of 0.7467 in the 70:30 split scenario without balancing. Logistic Regression produced a close weighted F1-score of 0.7462 and obtained the highest AUC value of 0.8010. Random Under Sampling improved hoax recall, but it also reduced the overall weighted F1-score.

Downloads

Download data is not yet available.

References

E. Zhuravskaya, M. Petrova, and R. Enikolopov, “Political Effects of the Internet and Social Media,” Annu. Rev. Econom., vol. 12, no. 1, pp. 415–438, Aug. 2020, doi: 10.1146/annurev-economics-081919-050239.

G. K. Shahi, A. Dirkson, and T. A. Majchrzak, “An exploratory study of COVID-19 misinformation on Twitter,” Online Soc. Netw. Media, vol. 22, no. May 2020, p. 100104, Mar. 2021, doi: 10.1016/j.osnem.2020.100104.

W. H. Organization, “Infodemic,” World Health Organization. Accessed: May 05, 2026. [Online]. Available: https://www.who.int/health-topics/infodemic

W. H. O. R. O. for Europe, “Infodemics and misinformation negatively affect people’s health behaviours, new WHO review finds,” World Health Organization. Accessed: May 05, 2026. [Online]. Available: https://www.who.int/europe/news/item/01-09-2022-infodemics-and-misinformation-negatively-affect-people-s-health-behaviours--new-who-review-finds

Kementerian Komunikasi dan Informatika Republik Indonesia, “Hingga akhir tahun 2023, Kominfo tangani 12.547 isu hoaks.” Accessed: May 09, 2025. [Online]. Available: https://www.komdigi.go.id/berita/pengumuman/detail/siaran-pers-no-02-hm-kominfo-01-2024-tentang-hingga-akhir-tahun-2023-kominfo-tangani-12-547-isu-hoaks

K. K. dan I. R. Indonesia, “Triwulan pertama 2023: Kominfo identifikasi 425 isu hoaks.” Accessed: May 09, 2025. [Online]. Available: https://www.komdigi.go.id/berita/siaran-pers/detail/siaran-pers-no-50-hm-kominfo-04-2023-tentang-triwulan-pertama-2023-kominfo-identifikasi-425-isu-hoaks

H. Oi-, Y. Li, E. Pastukhova, O. Brandts-, M. G. Tan, and M. G. Kirchhof, “YouTube as a source of misinformation on COVID- 19 vaccination?: a systematic analysis,” no. November 2021, pp. 1–6, 2022, doi: 10.1136/bmjgh-2021-008334.

L. Hickman, S. Thapa, L. Tay, M. Cao, and P. Srinivasan, “Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations,” Organ. Res. Methods, vol. 25, no. 1, pp. 114–146, Jan. 2022, doi: 10.1177/1094428120971683.

Z. Zhan, “Comparative Analysis of TF-IDF and Word2Vec in Sentiment Analysis: A Case of Food Reviews,” ITM Web of Conferences, vol. 70, p. 02013, Jan. 2025, doi: 10.1051/itmconf/20257002013.

N. E. Febriyanty, M. A. Hariyadi, and C. Crysdian, “Hoax Detection News Using Naïve Bayes and Support Vector Machine Algorithm,” International Journal of Advances in Data and Information Systems, vol. 4, no. 2, pp. 191–200, Oct. 2023, doi: 10.25008/ijadis.v4i2.1306.

B. Irena and Erwin Budi Setiawan, “Fake News (Hoax) Identification on Social Media Twitter using Decision Tree C4.5 Method,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 4, no. 4, pp. 711–716, Aug. 2020, doi: 10.29207/resti.v4i4.2125.

D. Ariatmanto and A. M. Rifai, “The Impact of Feature Extraction in Random Forest Classifier for Fake News Detection,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 8, no. 6, pp. 730–736, Dec. 2024, doi: 10.29207/resti.v8i6.6017.

A. R. I. Fauzy and Erwin Budi Setiawan, “Detecting Fake News on Social Media Combined with the CNN Methods,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 7, no. 2, pp. 271–277, Mar. 2023, doi: 10.29207/resti.v7i2.4889.

Y. Sibaroni, S. Mahadzir, S. S. Prasetiyowati, and A. F. Ihsan, “Combating Misinformation: Leveraging Deep Learning for Hoax Detection in Indonesian Political Social Media,” JURNAL INFOTEL, vol. 16, no. 2, pp. 413–426, May 2024, doi: 10.20895/infotel.v16i2.1139.

Muhammad Ikram Kaer Sinapoy, Yuliant Sibaroni, and Sri Suryani Prasetyowati, “Comparison of LSTM and IndoBERT Method in Identifying Hoax on Twitter,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 7, no. 3, pp. 657–662, 2023, doi: 10.29207/resti.v7i3.4830.

X. Zhou and R. Zafarani, “A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities,” ACM Comput. Surv., vol. 53, no. 5, 2021, doi: 10.1145/3395046.

H. Ahmed, I. Traore, and S. Saad, “Detecting opinion spams and fake news using text classification,” Security and Privacy, vol. 1, no. 1, pp. 1–15, 2018, doi: 10.1002/spy2.9.

Y. Yang, H. A. Khorshidi, and U. Aickelin, “A review on over-sampling techniques in classification of multi-class imbalanced datasets: insights for medical problems,” Front. Digit. Health, vol. 6, no. July, 2024, doi: 10.3389/fdgth.2024.1430245.

J. Hoyos-Osorio, A. Alvarez-Meza, G. Daza-Santacoloma, A. Orozco-Gutierrez, and G. Castellanos-Dominguez, “Relevant information undersampling to support imbalanced data classification,” Neurocomputing, vol. 436, pp. 136–146, 2021, doi: 10.1016/j.neucom.2021.01.033.

S. U. Hassan, J. Ahamed, and K. Ahmad, “Analytics of machine learning-based algorithms for text classification,” Sustainable Operations and Computers, vol. 3, no. March, pp. 238–248, 2022, doi: 10.1016/j.susoc.2022.03.001.

S. F. M. Radzi, M. K. A. Karim, M. I. Saripan, M. A. A. Rahman, I. N. C. Isa, and M. J. Ibahim, “Hyperparameter tuning and pipeline optimization via grid search method and tree-based autoML in breast cancer prediction,” J. Pers. Med., vol. 11, no. 10, 2021, doi: 10.3390/jpm11100978.

A. Tharwat, “Classification assessment methods,” Applied Computing and Informatics, vol. 17, no. 1, pp. 168–192, 2018, doi: 10.1016/j.aci.2018.08.003.

T. Fawcett, “An introduction to ROC analysis,” Pattern Recognit. Lett., vol. 27, no. 8, pp. 861–874, 2006, doi: 10.1016/j.patrec.2005.10.010.