This literature review explores the existing research on stock price prediction, focusing on the use of both deep learning and traditional models. By reviewing the current state of the art, this section aims to identify the strengths and limitations of each approach and highlight areas for further improvement. Understanding the theoretical foundations and empirical findings of these models is essential for developing a comprehensive framework for stock price prediction that leverages the strengths of both deep learning and traditional techniques.

The interest in stock market analysis is not limited to economics; it also attracts researchers from diverse backgrounds, including engineering disciplines. While most studies focus on stock market analysis, recent works have expanded to predicting forex and cryptocurrencies, reflecting the dynamic nature of financial markets [5] [6] . Cryptocurrencies, in particular, present unique challenges due to their decentralized structure and lack of regulatory oversight, making them more susceptible to manipulation and fraud compared to traditional stock markets.

Research has incorporated machine learning and more precisely deep learning for predicting the stock price and cryptocurrency. Previous research established that RNN effectively predicted daily Bitcoin prices better than other approaches such as ARIMA because of its capability to consider time-related characteristics of the data [7] . The studies conducted by the researchers have shown that although ARIMA generally presents high accuracy it needs stationary data as opposed to non-stationary time series handled by machine learning algorithms.

Also, a short-term stock prices forecasting model based was applied upon ARIMA, on three sectors based on the National Stock Exchange [8] . From their findings, the two identified possible gains in floating shares in the FMCG and Pharmaceuticals industries but possible loss in the Banking industry. In another study using data from Dhaka Stock Exchange was explored where they used a new univariate ANN model that overpowers the conventional ARIMA model for forecasting the total market capitalization analogously [9] .

Further contributing to the area of work, some work was done on the impact that hybrid models and gated recurrent units (GRUs) have in enhancing stock price prediction [10] [11] . Those approaches were extended the study by using a GRU-based model coupled with a procedure of dataset reconstruction to address overfitting issues [12] . These works demonstrate the effectiveness of implementing deep learning techniques in the stock market prediction despite there are some approaches that highlight that such patterns are not the only factors that must be taken into consideration for accurate prediction of the stock markets [13] [14] .

The problem of predicting stock prices has been deemed difficult due to the unstructured and non-stationary behavior of financial markets. Models based on statistical analysis have many drawbacks with regards to identifying the retrospective and probabilistic characteristics of patterns characteristic of stock markets. As for recent trends, authors have tried to use machine learning and deep learning to enhance the efficiency of stock price prediction.

Also, the similar line was employed by Nabipour et al. using decision trees, boosting techniques, and LSTM in an effort to classify the stock market group of Tehran Stock Exchange [15] . Their outcome showed that the LSTM model has better performance than the other techniques, which pointed out the use of deep learning in stock market prediction.

More recently, Liu et al., and Nikou et al. discovered that stock market indices and exchange-traded fund prices were better predicted by LSTM-based recurrent neural networks than simpler predictive models [2] [4] . These studies highlighted LSTM’s performance in modeling long-term dependencies in stock market data.

Moreover, Vijh et al. evaluated the factors of artificial neural network (ANN) and random forest (RF) and proved that ANN had better prediction performance than that of RF for some companies, which supported that ANN can be used for stock price prediction [3] . Hu et al. surveyed the literature regarding the recent increase in the use of LSTM fused with other deep learning techniques for financial forecasting and found that the use of deep learning methods is increasing continuously [16] .

Moghar et al. identified the application of LSTM in forecasting future stock market values, and this established the potential of recurrent neural networks in this field [17] . In a nutshell, the studies’ findings stress on the benefits of deep learning, especially LSTM, for improving the forecast of stock price and providing requisite information to investors and policymakers. Nti et al. conducted a comprehensive review of 122 papers published between 2007 and 2018 on stock market prediction [18] . They found that 66% of the papers used technical analysis methods, while 23% and 11% used fundamental analysis and combined analysis methods, respectively. Additionally, 98% of the fundamental analysis studies used data from social network sites to infer sentiment on financial markets. Commonly used technical indicators included SMA, EMA, MACD, RSI, and ROC.

The field of stock price prediction has seen significant advancements with the introduction of deep learning models, particularly in comparison to traditional models like autoregressive integrated moving averages (ARIMA) and artificial neural networks (ANN). This comparison has been explored in various studies, shedding light on the strengths and limitations of each approach.

Qihang compares three models: ARIMA, ANN, and LSTM networks, for stock price prediction [19] . The study suggests that LSTM may possess superior predictive abilities, contingent on effective data processing. Additionally, the ANN model outperforms the ARIMA model, highlighting the importance of incorporating time series data with external factors for improved predictions. Despite the advancements in LSTM models, all three models fundamentally derive predictions from potential relationships within time series data, neglecting external factors such as economic and political dynamics. The study emphasizes the need for future research to delve deeper into understanding and incorporating these external factors into predictive models for more accurate forecasting.

Seabe et al. focus on predicting cryptocurrency prices using deep learning models, including LSTM, Gated Recurrent Unit (GRU), and Bi-Directional LSTM (Bi-LSTM) [20] . The study concludes that accurate cryptocurrency predictions are crucial, emphasizing the challenge posed by the market’s nonlinearity. The evaluation identifies Bi-LSTM as the most effective model, outperforming LSTM and GRU. The study suggests that incorporating external factors such as social media and trading volumes could further enhance prediction accuracy. Additionally, a cost–benefit analysis acknowledges the expenses involved in building and maintaining prediction models but underscores the potential revenue generation and valuable predictions as significant benefits.

Both studies highlight the advancements made by deep learning models, particularly LSTM and Bi-LSTM, in stock price and cryptocurrency price prediction, respectively. These models have shown superior performance compared to traditional models like ARIMA and ANN. However, both studies also emphasize the importance of incorporating external factors into predictive models to improve accuracy further. While deep learning models have shown great promise, there is still ample room for further research and development to enhance their capabilities and address the complexities of financial markets.

The study aims at predicting the daily stock prices over a Five-year period starting from January 1, 2019, to December 31, 2023. The study tries to establish the models’ comparative performance in diverse scenarios by assessing their accuracy at various time frames, such as short-term (one year), mid-term (two and a half years), and long-term (five years).

Two goals of the project are to compare the accuracy of deep learning models with standard statistical models in order to produce helpful recommendations for risk managers and investors in Nigeria’s stock market. In as much as it seeks to identify the relative performance of these models there is a noble goal of providing direction to this decision making and improving investment strategies, thus the researchers’ desire of advancing the use of financial forecasting techniques. Figure 1 provides a clear and detailed illustration of how the deep learning method operates for stock prediction, aiding in a comprehensive understanding of its functionality.

The use of this methodological framework will lay a good platform for proper assessment of any of the selected models as will be seen in the subsequent analysis and comparison of their forecasting capability and reliability in the Nigerian stock market.

The dataset used in this study was obtained from reliable financial databases and covers daily stock prices for the specified period. It includes key parameters such as opening and closing prices, high and low prices, and trading volumes. To ensure the data’s accuracy and real-time updates, the study utilized the Yfinance API, which is known for providing comprehensive and dynamic financial data. This API forms a solid foundation for the analysis, crucial for the study’s success. The dataset’s reliability and comprehensiveness are vital, given its columns for Date, Open, High, Low, Close, Adjusted Close, and Volume. The Date column records the trading day, Open and Close prices reflect the stock’s prices at market opening and closing, High and Low indicate the day’s highest and lowest prices, Adjusted Close adjusts for corporate actions such as dividends, and Volume denotes the number of shares traded. Table 1 and Table 2 represents the pattern and summary of the data respectively.

The dataset consists of 1258 data points for each column. The mean values for the stock’s opening, highest, lowest, and closing prices are around 10.89, 11.03, 10.76, and 10.88, respectively. The mean adjusted close price is approximately 5.88, and the average trading volume is about 16892.58. The standard deviations indicate the degree of variation from the mean, with the highest variability observed in the trading volume.

The analysis in this study involves comparing the performance of deep learning models (LSTMs, GRUs) with traditional statistical models (ARMA, ARIMA) in predicting stock prices in the Nigerian stock market.

Long Short-Term Memory (LSTM) is a specialized form of recurrent neural network (RNN) designed to overcome the vanishing gradient problem, making it particularly effective at capturing long-term dependencies in sequential data such as stock prices. LSTMs are equipped with memory cells that can maintain information over long periods, enabling them to learn temporal patterns and relationships within the data. The LSTM architecture consists of several gates that regulate the flow of information:

Input Gate ( $i_t$): Determines how much of the new input to add to the cell state.

Forget Gate ( $f_t$): Decides what information to discard from the cell state.

Output Gate ( $o_t$): Controls the output and what information from the cell state to use for the hidden state.

The equations governing the behavior of an LSTM cell are as follows:

$\begin{array}{l}{i}_t=\sigma \left({\mathcal{W}}_{xi}{x}_t+{\mathcal{W}}_{hi}{h}_t+{\mathcal{W}}_{ci}{c}_{t-1}+1\right)\\ f_t=\sigma \left({\mathcal{W}}_{xf}{x}_t+{\mathcal{W}}_{hf}{h}_t+{\mathcal{W}}_{cf}{c}_{t-1}-b_f\right)\\ g_t=\tanh \left({\mathcal{W}}_{xi}{x}_t+{\mathcal{W}}_{hg}{h}_{t-1}+b_g\right)\\ o_t=\sigma \left({\mathcal{W}}_{xo}{x}_t+{\mathcal{W}}_{ho}{h}_{\mathcal{t}-1}+{\mathcal{W}}_{co}{c}_t+b_o\right)\\ c_t=f_t\odot c_{t-1}+i_t\odot g_t\\ h_t=o_t\odot \tanh \left(c_t\right)\end{array}$ (i)

where:

In the context of stock price prediction:

Gated Recurrent Unit (GRU) is a simplified version of the Long Short-Term Memory (LSTM) network, designed to achieve similar results with fewer parameters, making it computationally more efficient. GRUs merge the memory and input gates into a single structure, streamlining the learning process while still capturing essential temporal dependencies in sequential data like stock prices.

The GRU architecture includes two main gates:

The equations governing the GRU cell are:

$\begin{array}{l}{z}_t=\sigma \left({\mathcal{W}}_{xz}{x}_t+{\mathcal{W}}_{hz}{h}_{t-1}+b_z\right)\\ r_t=\sigma \left({\mathcal{W}}_{xr}{x}_t+{\mathcal{W}}_{hr}{h}_{t-1}+b_r\right)\\ {\tilde{h}}_t=\tanh \left({\mathcal{W}}_{xh}{x}_t+f_t\odot \left({\mathcal{W}}_{hh}{h}_{t-1}\right)+b_h\right)\\ h_t=\left(1-z_t\right)\odot h_{t-1}+z_t\odot {\tilde{h}}_t\end{array}$ (ii)

where:

In the context of stock price prediction:

The Autoregressive Moving Average (ARMA) model is a powerful statistical tool used to predict future values in a time series by combining autoregressive (AR) and moving average (MA) components. This model is particularly effective in capturing the linear dependencies in sequential data, making it suitable for stock price prediction.

The ARMA model comprises two key components:

Autoregressive (AR) Component: This part of the model expresses the current value of the series as a linear combination of its previous values. The order of the AR component is denoted by p.

$\text{AR}\left(p\right):y_t={\varphi }_1{y}_{t-1}+{\varphi }_2{y}_{t-2}+\cdots +{\varphi }_p{y}_{t-p}+{ϵ}_t$ (iii)

Moving Average (MA) Component: This part expresses the current value as a linear combination of past error terms. The order of the MA component is denoted by q.

$\text{MA}\left(q\right):y_t={ϵ}_t-{\theta }_1{ϵ}_{t-1}-{\theta }_2{ϵ}_{t-2}-\cdots -{\theta }_p{ϵ}_{t-q}$ (iv)

Therefore, the combined ARMA model is represented as $\text{ARMA}\left(p,q\right)$:

$y_t={\varphi }_1{y}_{t-1}+{\varphi }_2{y}_{t-2}+\cdots +{\varphi }_p{y}_{t-p}+{ϵ}_t-{\theta }_1{ϵ}_{t-1}-{\theta }_2{ϵ}_{t-2}-\cdots -{\theta }_p{ϵ}_{t-q}$ (v)

where:

In the context of stock price prediction, the ARMA model utilizes past stock prices and error terms to forecast future prices. This approach helps in understanding the underlying patterns in stock prices by accounting for both the linear relationships (AR part) and the shocks or unexpected changes (MA part).

The AutoRegressive Integrated Moving Average (ARIMA) model is an extension of the ARMA model designed to handle non-stationary time series data by incorporating differencing. This makes ARIMA particularly effective for predicting stock prices, where trends and seasonal patterns often cause non-stationarity.

The ARIMA model has three main components.

AutoRegressive (AR) Component: Like in the ARMA model, the AR component uses previous values to predict the current value. The order of this component is denoted by p. So, from equation (iii):

$\text{AR}\left(p\right):y_t={\varphi }_1{y}_{t-1}+{\varphi }_2{y}_{t-2}+\cdots +{\varphi }_p{y}_{t-p}+{ϵ}_t$

Integrated (I) Component: This part involves differencing the time series data to make it stationary. The order of differencing is denoted by d.

$\text{I}\left(d\right):{y^{\prime }}_t=y_t-y_{t-d}$ (vi)

Moving Average (MA) Component: Similar to the ARMA model, the MA component uses past error terms to predict the current value. The order of this component is denoted by q. So, from equation (iv):

$\text{MA}\left(q\right):y_t={\theta }_1{ϵ}_{t-1}+{\theta }_2{ϵ}_{t-2}+\cdots +{\theta }_p{ϵ}_{t-q}+{ϵ}_t$

Therefore, the combined ARMA model is represented as $\text{ARIMA}\left(p,d,q\right)$:

${y^{\prime }}_t={\varphi }_1{y^{\prime }}_{t-1}+{\varphi }_2{y^{\prime }}_{t-2}+\cdots +{\varphi }_p{y^{\prime }}_{t-p}+{\theta }_1{ϵ}_{t-1}+{\theta }_2{ϵ}_{t-2}+\cdots +{\theta }_p{ϵ}_{t-q}+{ϵ}_t$ (vii)

where:

In stock price prediction, the ARIMA model effectively captures both the linear relationships and the effects of trends and seasonality by differencing the data. This helps in stabilizing the time series and improving the accuracy of the forecasts.

To assess the accuracy and reliability of the models in predicting stock prices, this study employs several evaluation metrics. These metrics provide a quantitative basis for comparing the performance of deep learning models (LSTM, GRU) and traditional statistical models (ARMA, ARIMA). The metrics used include Mean Squared Error (MSE), Mean Absolute Percentage Error (MAPE), R-squared (R²), and Root Mean Squared Error (RMSE).

Mean Squared Error (MSE) measures the average squared difference between the predicted and actual stock prices. It penalizes larger errors more significantly, making it sensitive to outliers. MSE is given by:

$\text{MSE}=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{\left(P_i-A_i\right)}^2}$ (viii)

where:

In the context of stock prices, a lower MSE indicates that the model’s predictions are closer to the actual prices, suggesting better performance.

Mean Absolute Percentage Error (MAPE) measures the average absolute percentage difference between the predicted and actual stock prices. It provides a relative measure of prediction accuracy, making it easy to interpret. MAPE is given by:

$\text{MAPE}=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\left|\frac{P_i-A_i}{A_i}\right|}\times 100$ (ix)

MAPE is expressed as a percentage, with lower values indicating better predictive accuracy. It is particularly useful for comparing the performance of models across different datasets.

R-squared (R²) is a statistical measure that represents the proportion of the variance in the dependent variable (actual stock prices) that is predictable from the independent variable (predicted stock prices). It indicates the goodness of fit of the model. R² is given by:

${\text{R}}^2=1-\frac{{\displaystyle {\sum }_{i=1}^n{\left(A_i-P_i\right)}^2}}{{\displaystyle {\sum }_{i=1}^n{\left(A_i-{\bar{A}}_i\right)}^2}}$ (x)

where:

An R² value closer to 1 indicates that the model explains a large portion of the variance in the stock prices, signifying a good fit.

Root Mean Squared Error (RMSE) is the square root of the average squared difference between the predicted and actual stock prices. It is similar to MSE but is in the same units as the original data, making it more interpretable. RMSE is given by:

$\text{RMSE}=\sqrt{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{\left(P_i-A_i\right)}^2}}$ (xi)

Like MSE, a lower RMSE indicates better predictive performance. RMSE is particularly useful when the magnitude of the errors is important.

The analysis approach involves independently testing LSTM, GRU, ARMA, and ARIMA models on short-term (1 year), medium-term (2.5 years), and long-term (5 years) datasets. Figure 2 demonstrates the high-level abstraction of our analysis pattern. The Augmented Dickey-Fuller (ADF) test will be used to determine the stationarity of the data [21] . By assessing model performance across different time horizons, the study aims to provide insights into the models’ effectiveness under varying conditions. This progressive evaluation facilitates a comprehensive comparison, shedding light on the predictive accuracy of deep learning and statistical models over diverse time frames.

In the short-term analysis of stock price prediction models, the performance of four models was evaluated: ARIMA, ARMA, LSTM, and GRU. The R-squared values indicate the goodness of fit of each model, with higher values indicating a stronger correlation between predicted and actual values. In Table 3 , both ARIMA and ARMA models exhibited negative R-squared values, indicating poor performance compared to a simple mean model. This suggests that these traditional models were not effective in capturing the short-term movements of the stock price. On the other hand, the LSTM model showed impressive performance, with an R-squared value of 0.9885. This indicates a strong correlation between the predicted and actual values, suggesting that the LSTM model was highly effective in capturing short-term stock price movements. Similarly, the GRU model also performed well, with an R-squared value of 0.9796. This indicates that the GRU model was effective in capturing short-term stock price movements, although slightly less so than the LSTM model.

In terms of error metrics, both the LSTM and GRU models had considerably lower Root Mean Squared Error (RMSE), Mean Squared Error (MSE), and Mean Absolute Error (MAE) compared to the ARIMA and ARMA models. This further demonstrates the superior performance of the LSTM and GRU models in the short term.

Overall, the results suggest that the LSTM model was the best-performing model for short-term stock price prediction, followed closely by the GRU model. Moreover, Figure 5 shows the comparison of the predictions using different models that we discovered in short term analysis.

In the medium-term analysis of stock price prediction models, the performance of ARIMA, ARMA, LSTM, and GRU models was evaluated. In Table 4 , the ARIMA model showed a slight improvement in R-squared compared to the short term but still indicated poor performance. The R-squared value was −0.5725, suggesting that the ARIMA model was not effective in capturing medium-term stock price movements. The ARMA model performed significantly worse in the medium term, with a very low R-squared value of −10.9650. This indicates that the ARMA model is not suitable for medium-term stock price prediction.

In contrast, both the LSTM and GRU models performed well in the medium term. The LSTM model had an R-squared value of 0.8665, indicating a strong correlation between predicted and actual values. The GRU model performed slightly worse but still effectively captured medium-term stock price movements, with an R-squared value of 0.1256.

In terms of error metrics, both the LSTM and GRU models had lower RMSE, MSE, and MAE compared to the ARIMA and ARMA models, further confirming their effectiveness in the medium term. Overall, the results suggest that the LSTM model was the best-performing model for medium-term stock price prediction, followed by the GRU model. Furthermore, Figure 6 depicts the comparison of the predictions using different models that we discovered in medium term analysis.

In the long-term analysis of stock price prediction models, the performance of ARIMA, ARMA, LSTM, and GRU models was evaluated. In Table 5 , the ARIMA model exhibited an R-squared value of −2.3549, indicating poor performance in capturing long-term stock price movements. Similarly, the ARMA model performed poorly in the long term, with an R-squared value of −7.4547. In contrast, both the LSTM and GRU models showed much better performance in the long term. The LSTM model had an R-squared value of 0.4950, indicating a moderate correlation between predicted and actual values over the long term. The GRU model performed even better, with an impressive R-squared value of 0.9594, indicating a strong correlation between predicted and actual values in the long term. In terms of error metrics, both the LSTM and GRU models had lower RMSE, MSE, and MAE compared to the ARIMA and ARMA models, further confirming their effectiveness in the long term. Also, Figure 7 depicts the comparison of the predictions using different models that we discovered in long term analysis.

In summary, the results suggest that the GRU model was the best-performing model for long-term stock price prediction, followed by the LSTM model.