The formal analysis and forecasting of stock prices using rigorous mathematical models has surfaced since the last century. Traditionally, stock prediction utilized statistical methods such as stochastic calculus, Brownian motion, pricing models (e.g., Black-Scholes model, binomial options pricing model), and time series analysis. These methods analyze historical price data to identify trends or hidden patterns within the market. Despite their utility, traditional methods often fail to capture the non-linear relationships in stock markets. One such reason is that all these models require making simplistic assumptions, such as constant volatility and perfect information within historical price data, which is generally not true in the real world.

Machine learning, on the other hand, is more flexible and adapts to the given data set. Artificial intelligence (AI) techniques do not require overly simplistic assumptions and can take in any data, such as news sentiment information and any other relevant information, in addition to historical price data. Furthermore, artificial intelligence models have been experimentally proven to be more accurate than traditional statistical approaches [1]. Hence, the use of machine learning models in stock prediction has surged in popularity.

Linear models are one of the most common AI techniques applied in stock prediction. Linear regression models have demonstrated strong success for stock price prediction [2]. Other linear models, including support vector machine (SVC) models [3], Least Absolute Shrinkage and Selection Operator (LASSO) models [4][5], Ridge regression models [6], and gradient boosting machines [7], have been shown to surpass most traditional statistical methods in forecasting stock prices. In addition, decision tree models [8] and clustering methods [9] have produced results comparable to those of linear models. Extended models based on decision trees and clustering, such as the random forest model [10] and the K-nearest neighbor model [11], have shown even more potential in predicting stocks.

With the recent advancements in neural networks, the Long Short-Term Memory (LSTM) network has become one of the most popular models for stock prediction. Other types of neural networks have also gained popularity in the game of stock prediction, such as multi-layer perceptron (MLP) [12], convolution neural networks [13], recurrent neural networks (RNN) [14], artificial neural networks (ANN), and temporal convolutional networks (TCN) [15]. These models have shown similar results or even better results than simple linear models and decision trees.

The model proposed in this paper also utilizes stacked generalization, commonly known as stacking (Figure 1). Stacking is an ensemble modeling technique that effectively combines multiple base models to enhance predictive performance. Introduced in Wolpert (1992), stacking [16] involves training a meta-model that leverages the predicted outputs from these base models to generate a final prediction. This method has gained popularity for its ability to integrate the strengths of various models, allowing for more robust and accurate predictions across different applications.

Many ensemble methods (algorithms that combine multiple models) combine the power of numerous different algorithms, such as Weng et al. (2018) [17], Nti et al. (2020) [18], and Li and Pan (2021) [19]. Additionally, ensemble methods include other types of input data, such as financial news sentiment and other stock prices, such as news sentiment analysis in Halder (2022) [20], emotional factors in Zhang et al. (2024) [21], and gold prices.

These studies show the evolution of machine learning models in stock prediction, from linear models to ensemble models combining LSTMs and CNNs. This paper continues this development by proposing an ensemble model combining linear models and deep neural networks that take in sentiment analysis scores, fundamental analysis data, and technical indicators.

Figure 1. Flow of stacked generalization (i.e. stacking).

The proposed ensemble model contains 3 main parts: The input data integrating historical price data, financial analysis methods, and sentiment analysis scores, the hybrid model containing 14 different deep-learning algorithms, and an evaluation process.

The Mean Squared Error (MSE) results for the stock prediction using the proposed hybrid model on the following stocks over 50 days are shown in Table 2.

Mean Squared Error (MSE) is used to measure the average squared difference between the actual values (observed data) and the predicted values generated by a model. Mathematically, it is the average of the squared differences between predicted values and actual values for a dataset.

Table 3 shows the stock investment simulation results. It contains the stock invested, initial investment budget, return (or final capital), and the rate of return over 5 years.

Table 2. MSE for stock prediction.

Table 3. Return on investment for simulated stocks.

Table 4 shows the individual model performances on Google stock prices.

Table 4. Training and testing MSE for individual components with Google’s stock prices.

This TFS model demonstrated an improvement over individual models, achieving a lower Mean Squared Error (MSE) than all neural networks. However, compared to the lowest individual MSE of 0.15 for the KNN model, the hybrid model performs worse, likely influenced by additional complexity and non-linearity. This suggests that linear models, in this case, demonstrated stronger performance than neural networks, which may have introduced more complexity without significantly improving accuracy, thus slightly reducing accuracy in the hybrid model. One major issue with neural networks is that they have many more hyperparameters than simple linear models, and these hyperparameters are not optimized to produce the best prediction. If there were more computational power and time, neural networks would theoretically perform better than simple linear models due to their ability to capture non-linear relationships within the market.

Although the MSE of the TFS model is higher than the individual simple linear models, KNN, and decision tree models, the investment simulation using the TFS model and the KNN model both have identical returns at $4061 for Google. This is because the trading strategy employed is purely directional, relying on the binary decision to buy or sell.

In this context, the small difference in MSE (the magnitude of the error) does not affect the prediction enough to change the sign of the forecasted price. Therefore, both models made the exact same series of correct directional decisions throughout the simulation, resulting in identical practical performance and returns, despite having different statistical error metrics.

The performance of some models, particularly the SNN, also suffered due to untuned hyperparameters. The SNN model has an unusually high MSE at around 4000 for both training and testing. Hyperparameter optimization techniques, such as grid search or Bayesian optimization, a technique using a probabilistic model to guide the search, could improve model performance by finding the ideal parameter values. This would help balance the complexity and generalization of the models, ultimately reducing errors. The SNN is also one of the newer models developed without a prebuilt library to use; thus, coding the entire model from scratch may introduce a lot of errors, such as convergence issues and unoptimized architecture and hyperparameters. Therefore, it also introduces a lot of random noise into the final TFS model, decreasing its accuracy.

In addition, Bitcoin and the S&P 500 had an especially high MSE. This is likely due to the volatile nature of cryptocurrency, showing that predicting equity or cryptocurrency with high fluctuations in price is a much harder task, compared to more stable assets like Google or JPMorgan Chase stocks. However, index stocks are far less volatile than single stocks, so theoretically the TFS model should perform better at the S&P 500 as opposed to single stocks. However, the model performs significantly worse on index stocks. One potential explanation is that there is a lot more noise in the S&P 500 because the model is essentially trying to predict the combined movement of 500 major companies instead of 1 singular company, making the index value highly sensitive to broader macroeconomic trends that may not be reflected in individual firm’s stock prices. Therefore, although index funds are much more stable, their trends contain additional variables like high-level macroeconomic policies, such as change in interest rates or GDP, ultimately leading to a worse performance compared to single stocks predictions.

Moreover, the S&P 500 and Bitcoin do not have fundamental data. The model used the individual firm’s quarterly revenue, shareholder equity, operating cash flow, EPS, P/E ratio, and dividend yield which index funds and cryptocurrency data did not have. In addition, the S&P 500 news sentiment analysis is the collection of all news regarding the 503 companies, making each piece of news less important to the overall model, and it requires much more computational power to obtain all the news sources and their sentiment. Bitcoin’s speculative and unpredictable nature makes it even more difficult for traditional models to forecast accurately.

Regarding the input data, financial news sentiment analysis scores were not standard and hard to find. Much financial news data is not directly related to the stock being predicted, and some news is weighed more heavily than others. Moreover, using Large Language Models (LLMs) to score news sentiment introduces inherent errors and randomness. If one had access to all the available news sources about a specific company and could weigh each one according to how much it affects the stock market, sentiment analysis would contribute a lot more to the model, as opposed to news sentiment analysis scores not having much effect on the model. There was also a lack of sufficient and diverse data, particularly with natural language processing (NLP) inputs for sentiment analysis. NLP models tend to require large datasets to identify meaningful patterns, and in this case, the limited data led to higher error rates. At the same time, there was also insufficient data for news sentiment analysis to yield a concrete conclusion. The model should perform better if it had 20 - 30 years of data to train on, as opposed to 5-10 years of data.

One important takeaway from this analysis is the potential benefit of including more linear models and reducing reliance on neural networks (NNs). Linear models provide greater interpretability and stability, especially when working with datasets where relationships are more straightforward. Neural Networks, while powerful, can be prone to overfitting and may not always be the most effective choice for structured financial data. The large difference between the training MSE and testing MSE in Table 1 indicates overfitting in the model, despite using L2 regularization. It can also be shown from the individual model results that the linear models generally performed better than the neural networks. Therefore, a shift toward incorporating more linear models could yield more consistent results.

Using an autoencoder to preprocess the data turned out to make the model perform worse. A likely reason is that there is already too much noise in the stock market, and that extra noise hinders the performance of the model.

Looking ahead, incorporating additional data sources could enhance model performance. For instance, sentiment analysis from other news sources and social media could provide insights into market behavior, especially for volatile assets like Bitcoin. Similarly, macroeconomic indicators such as interest rates and inflation data could offer more context and improve predictions for traditional financial institutions. Additionally, more frequent data, such as the stock price at every hour instead of every day, could further refine predictions. In conclusion, the model would certainly benefit from an increased volume of data, more accurate data, and more computational power.

Finally, to improve the hybrid model further, weighted averaging could be used to give more importance to models that consistently perform well on specific subsets of the data. Overfitting should also be addressed by adding extra dropout layers and early stopping. Through careful tuning and combination of outputs, the hybrid model can evolve into an even more accurate and versatile prediction system.

In this study, we proposed the TriFusion Stacking (TFS) model, a hybrid approach that integrates fundamental data, technical indicators, and sentiment analysis to enhance stock price predictions. Our results show that the proposed TFS model is successful, as it has an MSE of less than 1 for predicting stock prices. However, using a standalone model of simple linear regression or K-nearest neighbor also produces a small MSE. Our findings highlight that linear models can sometimes outperform more complex neural networks when dealing with structured financial data. This suggests a need to re-evaluate the role of simpler models in ensemble strategies for stock prediction. Our findings also highlight that predicting the prices of volatile stocks like Bitcoin and index stocks like the S&P 500 is much more challenging. Future works in improving the model could include better quality news sentiment data and trying different meta-models (the model generating the final prediction, in this case, the MLP). Ultimately, our work contributes to the active field of stock price prediction by demonstrating the effectiveness of combining machine-learning algorithms and an enhanced understanding of which models are more important in stock price prediction and which stock types are more challenging to predict.

I would like to thank my mentor Odysseas Drosis for helping me with the model and guiding me through the writing process.