Exploring User Experience by User Review Using LDA-Topic  Modeling and HEART Framework: A Systematic Literature Review

Ayu Indriadika; Noviyanti Santoso

doi:10.57152/malcom.v5i4.2247

Authors

Ayu Indriadika Institute of Technology Sepuluh Nopember Surabaya
Noviyanti Santoso Institute of Technology Sepuluh Nopember Surabaya

DOI:

https://doi.org/10.57152/malcom.v5i4.2247

Keywords:

HEART, LDA, Machine Learning, Systematic Literature Review, User Experience, UX Evaluation

Abstract

This study aims to evaluate the integration of the HEART framework (Happiness, Engagement, Adoption, Retention, and Task Success) with computational modeling techniques such as Latent Dirichlet Allocation (LDA) for measuring User Experience (UX). A Systematic Literature Review (SLR) was conducted on articles published between 2015 and 2025, selected from reputable databases including Scopus. The selected studies emphasize the use of HEART metrics in conjunction with machine learning approaches, particularly LDA, and were assessed based on the Scimago journal quartile ranking system. The findings categorize the studies into five main research objectives: predicting user satisfaction and emotional response, optimizing usability, analyzing user-generated content, evaluating learning performance through gamified systems, and assessing system requirements in relation to UX. This classification reveals growing trends in applying hybrid methods that combine qualitative metrics with automated modeling techniques. The results underline the importance of developing more adaptive and scalable UX evaluation frameworks that align human-centered insights with machine learning-driven analysis. This study offers a foundational reference for future research in building integrative models that advance the depth and scale of UX assessments in complex digital environments.

Downloads

Download data is not yet available.

References

K. Rodden, H. Hutchinson, and X. Fu, “Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications,” in SIGCHI Conference on Human Factors in Computing Systems, 2010, pp. 2395–2398.

Z. Amiri et al., “The Personal Health Applications of Machine Learning Techniques in the Internet of Behaviors,” Sustainability, vol. 15, no. 16, pp. 1–41, Aug. 2023, doi: 10.3390/su151612406.

M. W. Iqbal, K. Shinan, S. R. Shahid Rafique, A. Alourani, M. U. Ashraf, and N. Z. A. Rahim, “Usability and Optimization of Online Apps in User’s Context,” PeerJ Comput. Sci., vol. 10, pp. 1–27, Dec. 2024, doi: 10.7717/peerj-cs.2561.

P. I. Santosa, “Measuring User Experience in an Online Store Pusing Pulse and Heart Metrics,” J. Ilm. Kursor, vol. 7, no. 3, pp. 145–154, 2014.

A. M. Mutawa and S. Sruthi, “Enhancing Human–Computer Interaction in Online Education: A Machine Learning Approach to Predicting Student Emotion and Satisfaction,” Int. J. Human–Computer Interact., vol. 40, no. 24, pp. 8827–8843, Dec. 2024, doi: 10.1080/10447318.2023.2291611.

D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet Allocation,” J. Mach. Learn. Res., vol. 3, pp. 993–1022, 2003.

H.-F. Lin, P.-C. Lin, and Be. Yeo, “Consumer-Generated Visual Advertisements in Social Media Brand Communities,” Int. J. Commun., vol. 18, pp. 982–1005, 2024.

M. Cheng and X. Jin, “What do Airbnb Users Care about? An Analysis of Online Review Comments,” Int. J. Hosp. Manag., vol. 76, pp. 58–70, Jan. 2019, doi: 10.1016/j.ijhm.2018.04.004.

M. Arief and N. A. Samsudin, “Neutral Class Handling for Customer Sentiment Analysis In Binary Classification: A Comparative Study of Supervised Machine Learning Classification Algorithm,” 2023 8th Int. Conf. Informatics Comput. ICIC 2023, vol. 1, pp. 1–8, 2023, doi: 10.1109/ICIC60109.2023.10381911.

E. Bigne, C. Ruiz, C. Perez-Cabañero, and A. Cuenca, Are customer star ratings and sentiments aligned? A deep learning study of the customer service experience in tourism destinations, vol. 17, no. 1. Springer Berlin Heidelberg, 2023. doi: 10.1007/s11628-023-00524-0.

K. Labille, S. Gauch, and S. Alfarhood, “Creating Domain-Specific Sentiment Lexicons via Text Mining,” IEEE Int. Conf. Progr. Compr., vol. 2022-March, pp. 36–47, 2022.

D. Monett and H. Stolte, “Predicting star ratings based on annotated reviews of mobile apps,” Proc. 2016 Fed. Conf. Comput. Sci. Inf. Syst. FedCSIS 2016, vol. 8, no. June 2015, pp. 421–428, 2016, doi: 10.15439/2016F141.

P. Rodrigues, I. S. Silva, G. A. R. Barbosa, F. R. dos S. Coutinho, and F. Mourão, “Beyond the Stars: Towards a Novel Sentiment Rating to Evaluate Applications in Web Stores of Mobile Apps,” in Proceedings of the 26th International Conference on World Wide Web Companion, in WWW ’17 Companion. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee, 2017, pp. 109–117. doi: 10.1145/3041021.3054139.

J. Sauro and J. R. Lewis, Quantifying the User Experience Practical Statistics for User Research. London: Morgan Kaufmann, 2016.

A. Garcia?Cabot, E. Garcia?Lopez, S. Caro?Alvaro, J. Gutierrez?Martinez, and L. De?Marcos, “Measuring the Effects on Learning Performance and Engagement with a Gamified Social Platform in an MSc Program,” Comput. Appl. Eng. Educ., vol. 28, no. 1, pp. 207–223, Jan. 2020, doi: 10.1002/cae.22186.

A. Hussain, E. O. C. Mkpojiogu, and F. M. Kamal, “The Role of Requirements in the Success or Failure of Software Projects,” Int. Rev. Manag. Mark., vol. 6, no. 7, pp. 306–311, 2016.

X. Liu, H. Shin, and A. C. Burns, “Examining the Impact of Luxury Brand’s Social Media Marketing on Customer Engagement: Using Big Data Analytics and Natural Language Processing,” J. Bus. Res., vol. 125, pp. 815–826, Mar. 2021, doi: 10.1016/j.jbusres.2019.04.042.

K. Ding, W. C. Choo, K. Y. Ng, and S. I. Ng, “Employing Structural Topic Modelling to Explore Perceived Service Quality Attributes in Airbnb Accommodation,” Int. J. Hosp. Manag., vol. 91, pp. 1–10, Oct. 2020, doi: 10.1016/j.ijhm.2020.102676.

Z. Wang, P. Gao, and X. Chu, “Sentiment Analysis from Customer-Generated Online Videos on Product Review Using Topic Modeling and Multi-Attention BLSTM,” Adv. Eng. Informatics, vol. 52, pp. 1–11, Apr. 2022, doi: 10.1016/j.aei.2022.101588.

B. Gao, M. Zhu, S. Liu, and M. Jiang, “Different Voices Between Airbnb and Hotel Customers: An Integrated Analysis of Online Reviews Using Structural Topic Model,” J. Hosp. Tour. Manag., vol. 51, pp. 119–131, Jun. 2022, doi: 10.1016/j.jhtm.2022.03.004.

B. Ozyurt and M. A. Akcayol, “A New Topic Modeling Based Approach for Aspect Extraction in Aspect Based Sentiment Analysis: SS-LDA,” Expert Syst. Appl., vol. 168, pp. 1–14, Apr. 2021, doi: 10.1016/j.eswa.2020.114231.

W. Li, L. Zhu, Y. Shi, K. Guo, and E. Cambria, “User Reviews: Sentiment Analysis using Lexicon Integrated Two-Channel CNN–LSTM Family Models,” Appl. Soft Comput., vol. 94, pp. 1–11, Sep. 2020, doi: 10.1016/j.asoc.2020.106435.

J. Hussain, Z. Azhar, H. F. Ahmad, M. Afzal, M. Raza, and S. Lee, “User Experience Quantification Model from Online User Reviews,” Appl. Sci., vol. 12, no. 13, pp. 1–24, Jul. 2022, doi: 10.3390/app12136700.

Y. Son and W. Kim, “Development of Methodology for Classification of User Eexperience (UX) in Online Customer Review,” J. Retail. Consum. Serv., vol. 71, pp. 1–13, Mar. 2023, doi: 10.1016/j.jretconser.2022.103210.

A. M. H. Abbas, K. I. Ghauth, and C.-Y. Ting, “User Experience Design Using Machine Learning: A Systematic Review,” IEEE Access, vol. 10, pp. 51501–51514, 2022, doi: 10.1109/ACCESS.2022.3173289.

X. Wang and B. Hu, “Machine Learning Algorithms for Improved Product Design User Experience,” IEEE Access, vol. 12, pp. 112810–112821, 2024, doi: 10.1109/ACCESS.2024.3442085.

R. Sadigov, E. Y?ld?r?m, B. Kocaç?nar, F. Patlar Akbulut, and C. Catal, “Deep Learning-Based User Experience Evaluation in Distance Learning,” Cluster Comput., vol. 27, no. 1, pp. 443–455, Feb. 2024, doi: 10.1007/s10586-022-03918-3.

C. I. Nwakanma, M. S. Hossain, J.-M. Lee, and D.-S. Kim, “Towards Machine Learning Based Analysis of Quality of User Experience (QoUE),” Int. J. Mach. Learn. Comput., vol. 10, no. 6, pp. 752–758, Dec. 2020, doi: 10.18178/ijmlc.2020.10.6.1001.

S. Chen et al., “LDA Ensembles for Interactive Exploration and Categorization of Behaviors,” IEEE Trans. Vis. Comput. Graph., vol. 26, no. 9, pp. 2775–2792, Sep. 2020, doi: 10.1109/TVCG.2019.2904069.

T. Ali, B. Omar, and K. Soulaimane, “Analyzing Tourism Reviews using an LDA Topic-Based Sentiment Analysis Approach,” MethodsX, vol. 9, pp. 1–10, 2022, doi: 10.1016/j.mex.2022.101894.

A. F. Pathan and C. Prakash, “Cross-Domain Aspect Detection and Categorization using Machine Learning for Aspect-based Opinion Mining,” Int. J. Inf. Manag. Data Insights, vol. 2, no. 2, pp. 1–18, Nov. 2022, doi: 10.1016/j.jjimei.2022.100099.

S. Kumi, C. Snow, R. K. Lomotey, and R. Deters, “Uncovering Concerns of Citizens Through Machine Learning and Social Network Sentiment Analysis,” IEEE Access, vol. 12, pp. 94885–94913, 2024, doi: 10.1109/ACCESS.2024.3426329.

H. He, G. Zhou, and S. Zhao, “Exploring E-Commerce Product Experience Based on Fusion Sentiment Analysis Method,” IEEE Access, vol. 10, pp. 110248–110260, 2022, doi: 10.1109/ACCESS.2022.3214752.

J. Dai, F. Lyu, L. Yu, Z. Zhou, and Y. He, “Medical Service Quality Evaluation Based on LDA and Sentiment Analysis: Examples of Seven Chronic Diseases,” Digit. Heal., vol. 10, pp. 1–20, Jan. 2024, doi: 10.1177/20552076241233864.

Y. Liu, F. Du, and H. Li, “The Impact of User Perception on Continued Knowledge Purchase Intention: A Study Utilizing the LDA Topic Analysis Method,” SAGE Open, vol. 15, no. 2, pp. 1–22, Apr. 2025, doi: 10.1177/21582440251339667.

S. Chekijian, H. Li, and S. Fodeh, “Emergency Care and the Patient Experience: Using Sentiment Analysis and Topic Modeling to Understand the Impact of the COVID-19 Pandemic,” Health Technol. (Berl)., vol. 11, no. 5, pp. 1073–1082, Sep. 2021, doi: 10.1007/s12553-021-00585-z.

J. Liu, P. Ding, and H. Jiang, “Exploring Sources of Patient Dissatisfaction in Mobile Health Communication: A Text Analysis Based on Structural Topic Model,” Digit. Heal., vol. 10, pp. 1–12, Jan. 2024, doi: 10.1177/20552076241287890.

H. Ozyurt, O. Ozyurt, and D. Mishra, “Exploring the Evolution of Educational Serious Games Research: A Topic Modeling Perspective,” IEEE Access, vol. 12, pp. 81827–81841, 2024, doi: 10.1109/ACCESS.2024.3411094.