Comparative Performance Study of Machine Learning Algorithms for Marketing Campaign Response Prediction
DOI:
https://doi.org/10.57152/ijatis.v3i2.3270Keywords:
Classification, Customer Response Prediction, Machine Learning, Marketing Analytics, Predictive AnalysisAbstract
This study presents a comparative analysis of five Machine Learning (ML) classification algorithms, namely Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), XGBoost, and Gradient Boosting Classifier (GBC), for predicting customer responses to marketing campaigns. The main challenge in this task is the highly imbalanced distribution of customer response data, which may cause biased model predictions. Therefore, this study evaluates model performance using precision, recall, and F1-score metrics to provide a more reliable assessment. Experiments were conducted using the Customer Personality Analysis dataset obtained from Kaggle. The results show that ensemble-based algorithms achieved more consistent performance than individual classifiers. XGBoost obtained the highest F1-score of 84.85% with an 80:20 data split, followed by Random Forest with 84.72%. In contrast, SVM showed lower stability, with its F1-score decreasing from 83.59% to 78.66% on the 90:10 data split. This study provides empirical evidence on the effectiveness of ensemble learning approaches, particularly XGBoost, in handling the complexity of customer behavior and supporting more targeted, data-driven marketing strategies.
References
M. El-Hajj and M. Pavlova, “Predictive Modeling of Customer Response to Marketing Campaigns,” Electronics (Switzerland), vol. 13, no. 19, Oct. 2024, doi: 10.3390/electronics13193953.
T. Davenport, A. Guha, D. Grewal, and T. Bressgott, “Cite this article 148k Accesses 141 Citations 33 Altmetric Metrics,” 2020, doi : 10.1007/s11747-019-00696-0.
B. Vla?i?, L. Corbo, S. Costa e Silva, and M. Dabi?, “The evolving role of artificial intelligence in marketing: A review and research agenda,” May 01, 2021, Elsevier Inc. doi: 10.1016/j.jbusres.2021.01.055.
McKinsey & Company, "What is personalization?," 2023. [Online]. Available: https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-personalization.
H. Langen and M. Huber, “How causal machine learning can leverage marketing strategies: Assessing and improving the performance of a coupon campaign,” arXiv preprint arXiv:2204.10820, 2022.
T. A. Smith, “The role of customer personality in satisfaction, attitude-to-brand and loyalty in mobile services,” Spanish Journal of Marketing - ESIC, vol. 24, no. 2, pp. 155–175, Oct. 2020, doi: 10.1108/SJME-06-2019-0036.
M. Syukron, R. Santoso, and T. Widiharih, “perbandingan metode smote random forest dan smote xgboost untuk klasifikasi tingkat penyakit hepatitis c pada imbalance class data”, [Online]. Available: https://ejournal3.undip.ac.id/index.php/gaussian/, 2020, doi: 10.14710/j.gauss.9.3.227-236.
I. Domor Mienye and Y. Sun, “A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects”, 2022, doi: 10.1109/ACCESS.2022.3207287.
A. Géron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 3rd ed. Sebastopol, CA, USA: O’Reilly Media, 2022.
Kulatilleke, G. K., & Samarakoon, S. (2022). Empirical study of machine learning classifier evaluation metrics behavior in massively imbalanced and noisy data. arXiv. https://doi.org/10.48550/arXiv.2208.11904
Opitz, J. (2024). A closer look at classification evaluation metrics and a critical reflection of common evaluation practice. Transactions of the Association for Computational Linguistics, 12, 1161–1183. https://doi.org/10.1162/tacl_a_00675
J. Lin, “Application of machine learning in predicting consumer behavior and precision marketing,” PLoS One, vol. 20, no. 5 May, May 2025, doi: 10.1371/journal.pone.0321854.
C. Ali, “klasifikasi topik berita politik menggunakan model logistic regression dan fitur bag of words,” Jurnal Kecerdasan Buatan dan Teknologi Informasi, vol. 4, no. 3, pp. 282–291, Sep. 2025, doi: 10.69916/jkbti.v4i3.375.
Y. Wang, W. Liao, H. Shen, Z. Jiang, and J. Zhou, “Some notes on the basic concepts of support vector machines,” J Comput Sci, vol. 82, Oct. 2024, doi: 10.1016/j.jocs.2024.102390.
S. A. Salleh et al., “Support Vector Machine (SVM) and Object Based Classification in Earth Linear Features Extraction: A Comparison,” Revue Internationale de Géomatique, vol. 33, no. 1, pp. 183–199, 2024, doi: 10.32604/rig.2024.050723.
I. T. Akinola, Y. Sun, I. G. Adebayo, and Z. Wang, “Daily peak demand forecasting using Pelican Algorithm optimised Support Vector Machine (POA-SVM),” Energy Reports, vol. 12, pp. 4438–4448, Dec. 2024, doi: 10.1016/j.egyr.2024.10.017.
Y. tao Zhu, C. shi Gu, and M. A. Diaconeasa, “A missing data processing method for dam deformation monitoring data using spatiotemporal clustering and support vector machine model,” Water Science and Engineering, vol. 17, no. 4, pp. 417–424, Dec. 2024, doi: 10.1016/j.wse.2024.08.003.
R. Chintalapudi, G. N. Kannaiyan, B. Pappula, and S. Makgato, “The novel Vogel’s approximation method integrated with a random forest algorithm in the vibration analysis of a two-directional functionally graded taper porous beam: Assessment,” Sci Afr, vol. 26, Dec. 2024, doi: 10.1016/j.sciaf.2024.e02397.
G. Samodra, Ngadisih, and F. S. Nugroho, “Benchmarking data handling strategies for landslide susceptibility modelingusing random forest workflows,” Artificial Intelligence in Geosciences, vol. 5, Dec. 2024, doi: 10.1016/j.aiig.2024.100093.
C. Bentéjac, A. Csörg?, and G. Martínez-Muñoz, “A comparative analysis of gradient boosting algorithms,” Artif Intell Rev, vol. 54, no. 3, pp. 1937–1967, Mar. 2021, doi: 10.1007/s10462-020-09896-5.
E. Gülter and M. F. Cevher, “Evolution of digital marketing campaigns with artificial intelligence and machine learning: Analysing success prediction capabilities,” Business & Management Studies: An International Journal, vol. 13, no. 2, pp. 478–493, Jun. 2025, doi: 10.15295/bmij.v13i2.2498.
Z. Liu, P. Jiang, K. W. De Bock, J. Wang, L. Zhang, and X. Niu, “Extreme gradient boosting trees with efficient Bayesian optimization for profit-driven customer churn prediction,” Technological Forecasting and Social Change, vol. 198, 2024, doi: 10.1016/j.techfore.2023.122945.
M. Mustak, H. Hallikainen, T. Laukkanen, L. Plé, L. D. Hollebeek, and M. Aleem, “Using machine learning to develop customer insights from user-generated content,” Journal of Retailing and Consumer Services, vol. 81, 2024, doi: 10.1016/j.jretconser.2024.104034.
A. Krawczyk, “Learning from imbalanced data: Open challenges and future directions,” Progress in Artificial Intelligence, vol. 12, pp. 1–23, 2023. doi: 10.1007/s13748-023-00295-5.
H. Chen, Q. Tang, Y. Wei, and M. Song, “Churn Prediction Model of Telecom Users Based on XGBoost,” Journal on Artificial Intelligence, vol. 3, no. 3, pp. 115–121, 2021, doi: 10.32604/jai.2021.026851.
U. Laili, Y. #1, A. Purwarianti, and I. +3, “JEPIN (Jurnal Edukasi dan Penelitian Informatika) Tuning Hyperparameter pada Gradient Boosting untuk Klasifikasi Soal Cerita Otomatis,” 2022, doi : 10.26418/jp.v8i1.50506.
Y. Hu, L. Tong, Y. Gao, Y. Zeng, B. Yan, and Z. Li, “GazeViT: A gaze-guided hybrid attention vision transformer for cross-view matching of street-to-aerial images,” Pattern Recognit Lett, vol. 191, pp. 80–88, May 2025, doi: 10.1016/j.patrec.2025.03.012.
P. Boozary, S. Sheykhan, H. GhorbanTanhaei, and C. Magazzino, “Enhancing customer retention with machine learning: A comparative analysis of ensemble models for accurate churn prediction,” International Journal of Information Management Data Insights, vol. 5, no. 1, Jun. 2025, doi: 10.1016/j.jjimei.2025.100331.
