An Improved Parallel Heterogeneous Long Short-Term Memory Model with Bayesian Optimization for Time Series Prediction

Tirupati G; Krishna Prasad MHM; Srinivasa Rao P

doi:10.52756/ijerr.2024.v45spl.009

Authors

Tirupati G Department of Information Technology, GVP College of Engineering for Women(A), India https://orcid.org/0000-0002-9160-7259
Krishna Prasad MHM Department of Computer Science & Engineering, College of Engineering (A), JNTUK, India https://orcid.org/0000-0003-2150-7807
Srinivasa Rao P Department of Computer Science & Engineering, MVGR College of Engineering (A), India https://orcid.org/0000-0003-4851-744X

DOI:

https://doi.org/10.52756/ijerr.2024.v45spl.009

Keywords:

Deep Learning, Hyperparameter, Optimization, Parallel LSTM, Prediction

Abstract

Currently, Deep Learning (DL) with the Recurrent Neural Networks (RNN) variants is being applied successfully in many domains of Engineering for prediction. In view of the demand for precise forecasting and the aid of Artificial Intelligence Tools, time series prediction reveals a vital task in decision-making and risk assessment. However, the application of novel Recurrent DL models for obtaining an accurate prediction of time series is yet to be explored. Recent trends reveal that Hybrid Neural Networks and DL models are appropriate for time series forecasts. At the same time, the model's selection and the hyperparameter's tuning can greatly impact its performance. To address this problem, a parallel long-term memory (PLSTM) model integrated with Bayesian hyperparameter optimization (PLSTM-BO) is proposed for time series prediction. The model is tuned in terms of key parameters, including the number of neurons, dropout, learning rate, and optimization technique. The model's performance is assessed using the SARS-COVID-19 cumulative cases, deaths, recovery cases, and NIFTY 50 stock closing price time series dataset. The obtained results convey that the current model exhibits remarkable performance compared to existing models.

References

Abbasimehr, H., & Paki, R. (2020). Prediction of COVID-19 Confirmed Cases Combining Deep Learning Methods and Bayesian Optimization. Chaos, Solitons & Fractals, 110511. https://doi.org/10.1016/j.chaos.2020.110511

Al-qaness, M. A. A., Ewees, A. A., Fan, H., & Abd El Aziz, M. A. E. (2020). Optimization Method for Forecasting Confirmed Cases of COVID-19 in China. Journal of Clinical Medicine, 9(3), 674. https://doi.org/10.3390/jcm9030674

Ardabili, S. F., Mosavi, A., Ghamisi, P., Ferdinand, F., Varkonyi- Koczy, A. R., Reuter, U., Rabczuk, T., & Atkinson, P.M. (2020). COVID-19 Outbreak Prediction with Machine Learning Algorithms, 13(10), 249. https://doi.org/10.3390/a13100249

Arora, P., Kumar, H., & Panigrahi, B. K. (2020). Prediction and analysis of COVID-19 positive cases using deep learning models: A descriptive case study of India. Chaos, Solitons & Fractals, 139, 110017. https://doi.org/10.1016/j.chaos.2020.110017

Bergstra, J., Remi, B., Yoshua, B., & Balazs, K. (2011). Algorithms for Hyper-Parameter Optimization, 25th Annual Conference on Neural Information Processing Systems.

Bemani, A., Baghban, A., Mosavi, A., & S., S. (2020). Estimating CO2-Brine diffusivity using hybrid models of ANFIS and evolutionary algorithms. Engineering Applications of Computational Fluid Mechanics, 14(1), 818–834. https://doi.org/ 10.1080/19942060.2020.1774422

Chandra, R., Jain, A., & Singh Chauhan, D. (2022). Deep learning via LSTM models for COVID-19 infection forecasting in India. PLOS ONE, 17(1), e0262708. https://doi.org/10.1371/journal.pone.0262708

Di Gennaro, F., Pizzol, D., Marotta, C., Antunes, M., Racalbuto, V., Veronese, N., & Smith, L. (2020). Coronavirus Diseases (COVID-19) Current Status and Future Perspectives: A Narrative Review. International Journal of Environmental Research and Public Health, 17(8), 2690. https://doi.org/10.3390/ijerph17082690

Global Development. (2021). Oxford Martin School. http://oxfordmartin.ox.ac.uk/global-development

Gorgolis, N., Hatzilygeroudis, I., Istenes, Z., & Gyenne, L. (2019). Hyperparameter Optimization of LSTM Network Models through Genetic Algorithm. 10th International Conference on Information, Intelligence, Systems and Applications (IISA). https://doi.org/10.1109/iis a.2019.8900675

Drewil, G. I., & Al-Bahadili, R. J. (2022). Air pollution prediction using LSTM deep learning and meta heuristics algorithms. Measurement: Sensors, 100546. https://doi.org/10.1016/j.measen.2022.100546

Graves, A. (2014). Generating sequences with recurrent neural networks. https://doi.org/10.48550/arXiv.1308.0850

Dubey, A., Dubey, S.M., Kumari, J., Yadav, P., & Sharma, G. (2024). Forecasting Wind Speed Using Clustering of Trend-Based Time Series Data. International Journal of Experimental Research and Review, 42, 40-49. https://doi.org/10.52756/ijerr.2024.v42.004

Haque, E., Tabassum, S., & Hossain, E. (2021). A Comparative Analysis of Deep Neural Networks for Hourly Temperature Forecasting. IEEE Access, 9,160646–160660. https://doi.org/10.1109/access.2021.3131533

Huang, R., Wei, C., Wang, B., Yang, J., Xu, X., Wu, S., & Huang, S. (2022). Well performance prediction based on Long Short-Term Memory (LSTM) neural network. Journal of Petroleum Science and Engineering, 208, 109686. https://doi.org/10.1016/j.petrol.2021.109686

Hochreiter, S., & Schmidhuber, J. (1997). Long Short Term Memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735

Hyperopt. (2019).hyperopt/hyperopt. Retrieved from https://github.com/hyperopt/hyperopt

K?rba?, I., Sozen, A., Tuncer, A. D., & Kazanc?oglu, F. ?. (2020). Comparative analysis and forecasting of COVID-19 cases in various European countries with ARIMA, NARNN and LSTM approaches. Chaos, Solitons& Fractals, 138, 110015. https://doi.org/10.1016/j.chaos.2020 .110015

Kumar, G., Singh, U. P., & Jain, S. (2022). An adaptive particle swarm optimization-based hybrid long short-term memory model for stock price time series forecasting. Soft Computing. https://doi.org/10.1007/s00500-022-07451-8

Li, X., Ma, X., Xiao, F., Xiao, C., Wang, F., & Zhang, S. (2022). Time-series production forecasting method based on the integration of Bidirectional Gated Recurrent Unit (Bi-GRU) network and Sparrow Search Algorithm (SSA). Journal of Petroleum Science and Engineering, 208, 109309. https://doi.org/10.1016/j.petrol.2021.109309

Malki, Z., Atlam, E.-S., Ewis, A., Dagnew, G., Alzighaibi, A. R., ELmarhomy, G., Elhosseini, M. A., Hassanien, A. E., & Gad, I. (2020). ARIMA models for predicting the end of COVID-19 pandemic and the risk of second rebound. Neural Computing and Applications, 33(7), 2929–2948. https://doi.org/10.1007/s00521-020-05434-0

Melin, P., Monica, J. C., Sanchez, D., & Castillo, O. (2020). Multiple Ensemble Neural Network Models with Fuzzy Response Aggregation for Predicting COVID-19 Time Series: The Case of Mexico. Healthcare, 8(2), 181. https://doi.org/10.3390/healthcare8020181

Moftakhar, L., Seif, M., & Safe, M. S. (2020). Exponentially Increasing Trend of Infected Patients with COVID-19 in Iran: A Comparison of Neural Network and ARIMA Forecasting Models. Iranian Journal of Public Health. https://doi.org/10.18502/ijph.v49is1.3675

Muhammed, M.O. (2023). Hyperparameter Optimization of a Parallelized LSTM for Time Series Prediction. 1–26. https://doi.org/10.1142/s2196888823500033

Rashid, T. A., Fattah, P., & Awla, D. K. (2018). Using Accuracy Measure for Improving the Training of LSTM with Metaheuristic Algorithms. Procedia Computer Science, 140, 324–333. https://doi.org/10.1016/j.procs.2018.10.307

Rustam, F., Reshi, A. A., Mehmood, A., Ullah, S., On, B., Aslam, W., & Choi, G. S. (2020). COVID-19 Future Forecasting Using Supervised Machine Learning Models. IEEE Access, 8, 101489–101499. https://doi.org/10.1109/ACCESS.2020.2997311

Shahid, F., Zameer, A., & Muneeb, M. (2020). Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM. Chaos, Solitons & Fractals, 140, 110212. https://doi.org/10.1016/j.chaos.2020.110212

Tirupati, G., Krishna Prasad, M.H.M., & Srinivasa Rao, P. (2021). COVID-19 Prediction Modeling Using Bidirectional Gated Recurrent Unit Network Model. Journal of Webology, 18(5), 15-41.

Wang, Y.W., Shen, Z. Z., & Jiang, Y. (2018). Comparison of ARIMA and GM (1,1) models for prediction of hepatitis B in China. PLOS ONE, 13(9), e0201987. https://doi.org/10.1371/journal.pone.0201987

Wang, P., Zheng, X., Ai, G., Liu, D., & Zhu, B. (2020). Time series prediction for the epidemic trends of COVID-19 using the improved LSTM deep learning method: Case studies in Russia, Peru & Iran. Chaos, Solitons & Fractals, 140, 110214. https://doi.org/10.1016/j.chaos.2020.110214

Willmott, C., & Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30, 79–82. https://doi.org/10.3354/cr030079

Wu, J., Chen, X.Y., Zhang, H., Xiong, L.D., Lei, H., & Deng, S.H. (2019). Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization. Journal of Electronic Science and Technology, 17(1), 26–40. https://doi.org/10.11989/JEST.1674-862X.80904120

Yahoo Finance. (2024). Yahoo Finance - Business Finance, Stock Market, Quotes, News. Yahoo Finance. https://finance.yahoo.com/

Yang, L., & Shami, A. (2020). On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing, 415, 295–316. https://doi.org/10.1016/j.neucom.2020.07.061

Yakubu, U. A., & Saputra, M. P. A. (2022). Time Series Model Analysis Using Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) for E-wallet Transactions during a Pandemic. International Journal of Global Operations Research, 3(3), 80–85. https://doi.org/10.47194/ijgor.v3i3.168

Zhou, Q., Zhou, C., & Wang, X. (2022). Stock prediction based on bidirectional gated recurrent unit with convolutional neural network and feature selection. PloS One, 17(2), e0262501. https://doi.org/10.1371/journal.pone.026250

Zivkovic, M., Bacanin, N., Venkatachalam, K., Nayyar, A., Djordjevic, A., Strumberger, I., & Al-Turjman, F. (2021). COVID-19 cases prediction by using hybrid machine learning and beetle antennae search approach. Sustainable Cities and Society, 66, 102669. https://doi.org/10.1016/j.scs.2020.102669