Optimasi Hyperparameter XGBoost Menggunakan Particle Swarm dan Grey Wolf Optimization untuk Prediksi Churn: XGBoost Hyperparameter Optimization Using Particle Swarm and Grey Wolf Optimization for Churn Prediction

Muhammad Farhan Gymnastiar; Diqy Fakhrun Shiddieq; Fikri Fahru Roji

doi:10.57152/malcom.v6i4.3049

Authors

Muhammad Farhan Gymnastiar Universitas Garut
Diqy Fakhrun Shiddieq Universitas Garut
Fikri Fahru Roji Universitas Garut

DOI:

https://doi.org/10.57152/malcom.v6i4.3049

Keywords:

Business_Impact, Churn_Prediction, GWO, PSO, XGBoost

Abstract

Customer churn merupakan tantangan kritis bagi industri berbasis langganan karena biaya akuisisi pelanggan baru jauh lebih tinggi dibandingkan biaya mempertahankan pelanggan yang sudah ada. Tujuan penelitian ini adalah membandingkan kinerja algoritma XGBoost dengan dua pendekatan optimasi hyperparameter, yaitu Particle Swarm Optimization (PSO) dan Grey Wolf Optimization (GWO), untuk memprediksi churn pelanggan pada layanan streaming. Dataset sintetis terkontrol sebanyak 10.000 sampel digunakan untuk memastikan perbandingan algoritma yang adil dan bebas dari kendala privasi. Evaluasi dilakukan melalui 10-fold cross-validation menggunakan metrik akurasi, recall, F1-score, dan AUC-ROC, serta dilengkapi dengan uji signifikansi statistik (paired t-test dan Wilcoxon) dan ablation test. Hasil menunjukkan bahwa kedua model yang dioptimalkan meningkatkan akurasi dibandingkan dengan baseline (baseline 96,26%; PSO 96,48%; GWO 96,52%). Uji statistik menunjukkan bahwa peningkatan yang dicapai XGBoost+PSO terhadap baseline bersifat signifikan (p = 0,0159), sedangkan peningkatan XGBoost+GWO belum signifikan (p = 0,0800). Analisis dampak bisnis menunjukkan bahwa penerapan model prediksi menghasilkan net benefit positif yang jauh lebih besar dibandingkan tanpa model. Temuan ini menegaskan bahwa optimasi metaheuristik, khususnya PSO, dapat memberikan peningkatan kinerja yang terukur, sekaligus menyoroti pentingnya validasi statistik dalam klaim superioritas algoritma. Penelitian lanjutan disarankan untuk menggunakan dataset riil guna menggeneralisasi temuan.

Downloads

Download data is not yet available.

References

M. R. Kurniawan, P. N. Sabrina, and R. Ilyas, “Prediksi Customer Churn Pada Perusahaan Telekomunikasi Menggunakan Algoritma C4.5 Berbasis Particle Swarm Optimization,” JATI (Jurnal Mhs. Tek. Inform., 2024.

A. Muneer, R. F. Ali, A. Alghamdi, S. M. Taib, A. Almaghthawi, and E. A. A. Ghaleb, “Predicting customers churning in banking industry: A machine learning approach,” Indones. J. Electr. Eng. Comput. Sci., vol. 26, no. 1, p. 539, 2022, doi: 10.11591/ijeecs.v26.i1.pp539-549.

T. Rashid, L. Li, and W. Wei, “Customer churn prediction in telecommunication using XGBoost,” IEEE Access, vol. 7, pp. 95127–95134, 2019, doi: 10.1109/ACCESS.2019.2928353.

D. A. Ardhani and K. D. Tania, “Knowledge discovery on e-commerce customer churn using interpretable machine learning: A comparative study of SHAP-based classifiers,” J. Appl. Informatics Comput., vol. 9, no. 5, pp. 2695–2702, 2025, doi: 10.30871/jaic.v9i5.10811.

L. Saha, H. K. Tripathy, S. R. Nayak, A. K. Bhoi, and P. Barsocchi, “Enhancing customer retention in telecom industry with machine learning driven churn prediction,” Sci. Rep., vol. 14, 2024, doi: 10.1038/s41598-024-63750-0.

B. Huang, M. T. Kechadi, and B. Buckley, “Customer churn prediction in telecommunications,” Expert Syst. Appl., vol. 39, no. 1, pp. 1414–1425, 2012, doi: 10.1016/j.eswa.2011.08.024.

T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2016, pp. 785–794. doi: 10.1145/2939672.2939785.

A. K. Ahmad, A. Jafar, and K. Aljoumaa, “Customer churn prediction in telecom using machine learning in big data platform,” J. Big Data, vol. 6, 2019, doi: 10.1186/s40537-019-0191-6.

B. Elnaim, “An efficient churn prediction model using gradient boosting machine and metaheuristic optimization,” Sci. Rep., vol. 13, 2023, doi: 10.1038/s41598-023-41093-6.

J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proc. IEEE Int. Conf. Neural Networks, 1995, pp. 1942–1948. doi: 10.1109/ICNN.1995.488968.

S. Mirjalili, S. M. Mirjalili, and A. Lewis, “Grey wolf optimizer,” Adv. Eng. Softw., vol. 69, pp. 46–61, 2014, doi: 10.1016/j.advengsoft.2013.12.007.

D. Pebrianti, H. Kurniawan, L. Bayuaji, and R. Rusdah, “XGBoost hyper-parameter tuning using particle swarm optimization for stock price forecasting,” J. Ilm. Tek. Elektro Komput. dan Inform., vol. 9, no. 4, pp. 1179–1195, 2024, doi: 10.26555/jiteki.v9i4.27712.

T. Vafeiadis, K. I. Diamantaras, G. Sarigiannidis, and K. C. Chatzisavvas, “A comparison of machine learning techniques for customer churn prediction,” Simul. Model. Pract. Theory, vol. 55, pp. 1–9, 2015, doi: 10.1016/j.simpat.2015.03.003.

I. Ullah, B. Raza, A. K. Malik, M. Imran, S. U. Islam, and S. W. Kim, “A churn prediction model using random forest: Analysis of machine learning techniques for churn prediction and factor identification in telecom sector,” IEEE Access, vol. 7, pp. 60134–60149, 2019, doi: 10.1109/ACCESS.2019.2914999.

S. Saha, C. Saha, M. M. Haque, M. G. R. Alam, and A. Talukder, “ChurnNet: Deep learning enhanced customer churn prediction in telecommunication industry,” IEEE Access, vol. 12, pp. 4471–4484, 2024, doi: 10.1109/ACCESS.2024.3349950.

A. De Caigny, K. Coussement, and K. W. De Bock, “A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees,” Eur. J. Oper. Res., vol. 269, no. 2, pp. 760–772, 2018, doi: 10.1016/j.ejor.2018.02.009.

V. Chang, K. Hall, Q. A. Xu, F. O. Amao, M. A. Ganatra, and V. Benson, “Prediction of customer churn behavior in the telecommunication industry using machine learning models,” Algorithms, vol. 17, no. 6, p. 231, 2024, [Online]. Available: https://www.mdpi.com/1999-4893/17/6/231

M. Z. Alotaibi and M. A. Haq, “Customer Churn Prediction for Telecommunication Companies using Machine Learning and Ensemble Methods,” Eng. Technol. Appl. Sci. Res., vol. 14, no. 3 SE-, pp. 14572–14578, Jun. 2024, doi: 10.48084/etasr.7480.

A. Manzoor, M. A. Qureshi, E. Kidney, and L. Longo, “A review on machine learning methods for customer churn prediction and recommendations for business practitioners,” IEEE Access, vol. 12, pp. 70434–70463, 2024, doi: 10.1109/ACCESS.2024.3402092.

J. Cai, “The causes of bank customer churn based on XGBoost and LightGBM models: The evidence from the Kaggle Dataset,” Financ. Econ., vol. 1, no. 4, 2024, doi: 10.61173/ggw4ga77.

K. G. Abay and Y. Yiblet, “A comparative study between Gray Wolf and particle swarm algorithms use for optimization of cost in composite beam,” Soft Comput., 2024, doi: 10.1007/s00500-023-09560-4.

P. Orzechowski and J. H. Moore, “Generative and reproducible benchmarks for comprehensive evaluation of machine learning classifiers,” Sci. Adv., vol. 8, no. 47, 2022, doi: 10.1126/sciadv.abl4747.

D. Dua and C. Graff, “UCI machine learning repository,” 2019, University of California, Irvine, School of Information and Computer Sciences. [Online]. Available: https://archive.ics.uci.edu/ml

G. Li, Y. Cui, and J. Su, “Adaptive mechanism-based grey wolf optimizer for feature selection in high-dimensional classification,” PLoS One, vol. 20, no. 5, p. e0318903, 2025, doi: 10.1371/journal.pone.0318903.

L. Yang and A. Shami, “On hyperparameter optimization of machine learning algorithms: Theory and practice,” Neurocomputing, vol. 415, pp. 295–316, 2020, doi: 10.1016/j.neucom.2020.07.061.

P. Probst, A.-L. Boulesteix, and B. Bischl, “Tunability: Importance of hyperparameters of machine learning algorithms,” J. Mach. Learn. Res., vol. 20, no. 53, pp. 1–32, 2019.

F. Pedregosa, “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011.

F. F. Reichheld, “The value of keeping the right customers,” Harv. Bus. Rev., 2014, [Online]. Available: https://hbr.org/2014/10/the-value-of-keeping-the-right-customers

F. F. Reichheld and W. E. Sasser, “Zero defections: Quality comes to services,” Harv. Bus. Rev., vol. 68, no. 5, pp. 105–111, 1990.

A. S. Bhandari and A. Kumar, “Grey wolf optimizer and hybrid PSO-GWO for reliability optimization and redundancy allocation problem,” Qual. Reliab. Eng. Int., 2023, doi: 10.1002/qre.3265.