Implementation of Gated Recurrent Unit, Long Short-Term Memory and Derivatives for Gold Price Prediction
DOI:
https://doi.org/10.57152/predatecs.v2i2.1609Keywords:
Gated Recurrent Unit, Gold, Long Short-Term Memory, PredictionAbstract
Gold is a precious metal with high resale value, often considered a safe investment as its price typically rises with inflation, attracting investors. However, even slight changes in gold prices can have significant impacts. To build an accurate forecasting model, this study applies and compares Gated Recurrent Unit (GRU), Bidirectional Gated Recurrent Unit (Bi-GRU), Long Short-Term Memory (LSTM), and Bidirectional Long Short-Term Memory (Bi-LSTM) algorithms on global gold prices. GRU and LSTM are recurrent neural networks designed to capture patterns in sequential data, where GRU uses a simplified gating mechanism to retain essential information, and LSTM, with its more complex gates, helps manage long-term dependencies in data. Bi-GRU and Bi-LSTM process data bidirectionally, capturing context from both past and future sequences for better prediction accuracy. This research uses data from Yahoo Finance (01-01-2014 to 12-06-2024) and experiments with optimization techniques (Adam, AdamW, Adamax, and Nadam), batch sizes (8, 16, and 32), time steps (10, 20, and 30), and a learning rate of 0.0001, trained for 1000 epochs with checkpoints and early stopping. Bi-GRU with Nadam, batch size 8, and 20 time steps proved most effective, with MSE of 4.1153, RMSE of 2.0286, MAE of 1.5881, and MAPE of 0.8857%. Forecasts using this model predict a 20-day decline in gold prices.
References
R. C. Staudemeyer and E. R. Morris, “Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks,” Sep. 2019, [Online]. Available: http://arxiv.org/abs/1909.09586
“The Case for Gold | World Gold Council.” Accessed: Nov. 16, 2024. [Online]. Available: https://invest.gold/why-invest-in-gold
J. Chai, C. Zhao, Y. Hu, and Z. G. Zhang, “Structural analysis and forecast of gold price returns,” Journal of Management Science and Engineering, vol. 6, no. 2, pp. 135–145, Jun. 2021, doi: 10.1016/j.jmse.2021.02.011.
G. D. Mishra et al., “EMAS position statement: Predictors of premature and early natural menopause,” Maturitas, vol. 123, pp. 82–88, May 2019, doi: 10.1016/j.maturitas.2019.03.008.
P. S. A. Surya Pangestu, A. Rochman, and A. Zuhdi, “Comparison of Gold Price Prediction Techniques Using Long Short Term Memory (LSTM) And Fuzzy Time Series (FTS) Method,” Intelmatics, vol. 3, no. 2, pp. 57–62, Aug. 2023, doi: 10.25105/itm.v3i2.17325.
I. E. Livieris, E. Pintelas, and P. Pintelas, “A CNN–LSTM model for gold price time-series forecasting,” Neural Comput Appl, vol. 32, no. 23, pp. 17351–17360, Dec. 2020, doi: 10.1007/s00521-020-04867-x.
I. Banerjee et al., “Comparative effectiveness of convolutional neural network (CNN) and recurrent neural network (RNN) architectures for radiology text report classification,” Artif Intell Med, vol. 97, pp. 79–88, Jun. 2019, doi: 10.1016/j.artmed.2018.11.004.
Y. Wang, W. Liao, and Y. Chang, “Gated Recurrent Unit Network-Based Short-Term Photovoltaic Forecasting,” Energies (Basel), vol. 11, no. 8, p. 2163, Aug. 2018, doi: 10.3390/en11082163.
T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA: ACM, Aug. 2016, pp. 785–794. doi: 10.1145/2939672.2939785.
M. G. Sobol, “Panel Mortality and Panel Bias,” J Am Stat Assoc, vol. 54, no. 285, p. 52, Mar. 1959, doi: 10.2307/2282139.
F. Meng, T. Song, D. Xu, P. Xie, and Y. Li, “Forecasting tropical cyclones wave height using bidirectional gated recurrent unit,” Ocean Engineering, vol. 234, p. 108795, Aug. 2021, doi: 10.1016/j.oceaneng.2021.108795.
S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput, vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997, doi: 10.1109/78.650093.
S. Ruder, “An overview of gradient descent optimization algorithms,” CoRR, pp. 1–14, Sep. 2016.
D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” ICLR, pp. 1–15, Dec. 2014.
I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” ICLR, pp. 1–19, Nov. 2017.
P. Radhakrishnan and G. Senthilkumar, “Nesterov-accelerated Adaptive Moment Estimation NADAM-LSTM based text summarization1,” Journal of Intelligent & Fuzzy Systems, vol. 46, no. 3, pp. 6781–6793, Mar. 2024, doi: 10.3233/JIFS-224299.
A. Rahmadeyan and Mustakim, “Long Short-Term Memory and Gated Recurrent Unit for Stock Price Prediction,” in Procedia Computer Science, Elsevier B.V., 2024, pp. 204–212. doi: 10.1016/j.procs.2024.02.167.
M. F. Fayyad, V. Kurniawan, M. R. Anugrah, B. H. Estanto, and T. Bilal, “Application of Recurrent Neural Network Bi-Long Short-Term Memory, Gated Recurrent Unit and Bi-Gated Recurrent Unit for Forecasting Rupiah Against Dollar (USD) Exchange Rate,” Public Research Journal of Engineering, Data Technology and Computer Science, vol. 2, no. 1, pp. 1–10, Apr. 2024, doi: 10.57152/predatecs.v2i1.1094.
X. Li, X. Ma, F. Xiao, C. Xiao, F. Wang, and S. Zhang, “Time-series production forecasting method based on the integration of Bidirectional Gated Recurrent Unit (Bi-GRU) network and Sparrow Search Algorithm (SSA),” J Pet Sci Eng, vol. 208, p. 109309, Jan. 2022, doi: 10.1016/j.petrol.2021.109309.
M. Yurtsever, “Gold Price Forecasting Using LSTM, Bi-LSTM and GRU,” European Journal of Science and Technology, vol. 31, no. 1, pp. 341–347, Dec. 2021, doi: 10.31590/ejosat.959405.
H. Crisóstomo de Castro Filho et al., “Rice Crop Detection Using LSTM, Bi-LSTM, and Machine Learning Models from Sentinel-1 Time Series,” Remote Sens (Basel), vol. 12, no. 16, p. 2655, Aug. 2020, doi: 10.3390/rs12162655.
F. Shahid, A. Zameer, and M. Muneeb, “Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM,” Chaos Solitons Fractals, vol. 140, p. 110212, Nov. 2020, doi: 10.1016/j.chaos.2020.110212.
Yahoo Finance, “Historical Gold Prices.”
M. Javed Awan, M. Shafry Mohd Rahim, H. Nobanee, A. Munawar, A. Yasin, and A. Mohd Zain Azlanmz, “Social Media and Stock Market Prediction: A Big Data Approach,” Computers, Materials & Continua, vol. 67, no. 2, pp. 2569–2583, 2021, doi: 10.32604/cmc.2021.014253.
A. Boukerche and J. Wang, “Machine Learning-based traffic prediction models for Intelligent Transportation Systems,” Computer Networks, vol. 181, p. 107530, Nov. 2020, doi: 10.1016/j.comnet.2020.107530.
J. Awwalu and F. Ogwueleka, “On Holdout and Cross Validation: A Comparison between Neural Network and Support Vector Machine,” International Journal of Trend in Research and Development, vol. 6, no. 2, pp. 2394–9333, Jun. 2019.
N. Zhai, P. Yao, and X. Zhou, “Multivariate Time Series Forecast in Industrial Process Based on XGBoost and GRU,” in 2020 IEEE 9th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), IEEE, Dec. 2020, pp. 1397–1400. doi: 10.1109/ITAIC49862.2020.9338878.
S. B. Primananda and S. M. Isa, “Forecasting Gold Price in Rupiah using Multivariate Analysis with LSTM and GRU Neural Networks,” Advances in Science, Technology and Engineering Systems Journal, vol. 6, no. 2, pp. 245–253, Mar. 2021, doi: 10.25046/aj060227.
L. Bi, G. Hu, M. M. Raza, Y. Kandel, L. Leandro, and D. Mueller, “A Gated Recurrent Units (GRU)-Based Model for Early Detection of Soybean Sudden Death Syndrome through Time-Series Satellite Imagery,” Remote Sens (Basel), vol. 12, no. 21, p. 3621, Nov. 2020, doi: 10.3390/rs12213621.
F. Meng, T. Song, D. Xu, P. Xie, and Y. Li, “Forecasting tropical cyclones wave height using bidirectional gated recurrent unit,” Ocean Engineering, vol. 234, pp. 1–11, Aug. 2021, doi: 10.1016/j.oceaneng.2021.108795.
P. Li et al., “Bidirectional Gated Recurrent Unit Neural Network for Chinese Address Element Segmentation,” ISPRS Int J Geoinf, vol. 9, no. 11, p. 635, Oct. 2020, doi: 10.3390/ijgi9110635.
N. T. Luchia, E. Tasia, I. Ramadhani, A. Rahmadeyan, and R. Zahra, “Performance Comparison Between Artificial Neural Network, Recurrent Neural Network and Long Short-Term Memory for Prediction of Extreme Climate Change,” Public Research Journal of Engineering, Data Technology and Computer Science, vol. 1, no. 2, pp. 62–70, Feb. 2024, doi: 10.57152/predatecs.v1i2.864.
A. Alsharef, K. Aggarwal, Sonia, M. Kumar, and A. Mishra, “Review of ML and AutoML Solutions to Forecast Time-Series Data,” Archives of Computational Methods in Engineering, vol. 29, no. 7, pp. 5297–5311, Nov. 2022, doi: 10.1007/s11831-022-09765-0.
S. Suryadibrata, “Gold Price Prediction in COVID-19 Era,” International Journal of Computational Intelligence in Control Copyrights @Muk Publications, vol. 13, no. 2, pp. 1–4, 2021.
N. N. Aung, J. Pang, M. C. H. Chua, and H. X. Tan, “A novel bidirectional LSTM deep learning approach for COVID-19 forecasting,” Sci Rep, vol. 13, no. 1, p. 17953, Oct. 2023, doi: 10.1038/s41598-023-44924-8.
J. Yin, Z. Deng, A. V. M. Ines, J. Wu, and E. Rasu, “Forecast of short-term daily reference evapotranspiration under limited meteorological variables using a hybrid bi-directional long short-term memory model (Bi-LSTM),” Agric Water Manag, vol. 242, Dec. 2020, doi: 10.1016/j.agwat.2020.106386.
S. Aryal, D. Nadarajah, D. Kasthurirathna, L. Rupasinghe, and C. Jayawardena, “Comparative analysis of the application of Deep Learning techniques for Forex Rate prediction,” in 2019 International Conference on Advancements in Computing (ICAC), IEEE, Dec. 2019, pp. 329–333. doi: 10.1109/ICAC49085.2019.9103428.
C. Arthur, N. Yudistira, and C. Dewi, “AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction,” IEEE Access, vol. 12, pp. 14014–14026, 2024, doi: 10.1109/ACCESS.2024.3356553.
R. Llugsi, S. El Yacoubi, A. Fontaine, and P. Lupera, “Comparison between Adam, AdaMax and Adam W optimizers to implement a Weather Forecast based on Neural Networks for the Andean city of Quito,” in 2021 IEEE Fifth Ecuador Technical Chapters Meeting (ETCM), IEEE, Oct. 2021, pp. 1–6. doi: 10.1109/ETCM53643.2021.9590681.
C. Christodoulos, C. Michalakelis, and D. Varoutas, “On the combination of exponential smoothing and diffusion forecasts: An application to broadband diffusion in the OECD area,” Technol Forecast Soc Change, vol. 78, no. 1, pp. 163–170, Jan. 2011, doi: 10.1016/j.techfore.2010.08.007.
X. Zhang, T. Zhang, A. A. Young, and X. Li, “Applications and Comparisons of Four Time Series Models in Epidemiological Surveillance Data,” PLoS One, vol. 9, no. 2, p. e88075, Feb. 2014, doi: 10.1371/journal.pone.0088075.
Z. Hu, Y. Zhao, and M. Khushi, “A Survey of Forex and Stock Price Prediction Using Deep Learning,” Applied System Innovation, vol. 4, no. 1, p. 9, Feb. 2021, doi: 10.3390/asi4010009.
Suryani and Mustakim, "An Intelligent Chatbot for Faculty Administration Using Bidirectional LSTM and Seq2Seq Architecture," 2024 International Conference on Smart Computing, IoT and Machine Learning (SIML), Surakarta, Indonesia, 2024, pp. 226-231, doi: 10.1109/SIML61815.2024.10578161.
