This is a special time because of COVID. As cases and deaths surge across the globe, people are confronting uncertainty through and beyond the global health crisis. This uncertainty may cause anxiety, fear, and other irrational reactions from the general public. Because of the sensitivity, concern, and other difficulties including more restrictive policies and shutdowns, stock markets are affected in an unrepresented way. This instability caused a great loss for investors, more fragile and volatile price returns and stock performance, and other social problems including wealth inequality [1]. Thus, improving the performance of models aiming to analyze and predict stock prices in the market is meaningful for both private investors and the public interest.

Predicting the stock price change is a challenging task, involving various factors and non-linear relationships among them [2] [3]. Some of these factors are not easily understood and quantifiable, like people’s subjective emotions. Things become more challenging and volatile after the pandemic. But even with these difficulties, stock price prediction is a trending topic.

Since the outbreak of the pandemic, there is an increasing number of research analyzing the connection between COVID-19 and the movements in the capital market [4] [5] [6]. More specifically, most of these studies utilize statistical analysis to anticipate COVID-19 data and leading pharmaceutical stock performance [4] [5]. There are some reviews that explore the potential of machine learning models by examining the correlation between COVID-19 statistics and general stock market performance [6] [7]. For instance, Adekoya and his team employed decision tree models to examine the correlation. And Rouf’s team focused more on the Indian general stock market.

Nonetheless, these studies and research are different from the metric and goal of our research topic, which emphasizes more about whether COVID-19 statistics can help improve the prediction performance of machine learning models in terms of future pharmaceutical stock price directions. In this study, we mainly focus on two research questions:

1) How to predict the next-day direction of pharmaceutical companies’ stock prices movement using a machine learning approach?

2) During the pandemic, to what extent could external information, especially COVID-related features improve the performance of the machine learning models that this study has developed?

Our research intends to fill the research gap by examining whether the inclusion of COVID-related data and some other stock environmental data in the features with prepossessing during pandemics will increase the performance of different machine learning models.

The main contributions of our paper can be summarized as follows:

1) We collected detailed company-specific financial statistics, including current and previous stock prices and key financial measuring metrics, selected global industry indexes, and the COVID-related data. We combined all these data together and transformed them into a single-day oriented dataset that can be used to train various models;

2) We developed a series of handcrafted features for COVID-related data and selected global industry indexes to aid the prediction of stock price change direction. The results suggest an average of 1.7% increase in AUROC with the inclusion of industry indexes features and an average of 6.5% increase in AUROC with the introduction of COVID-related features;

3) We further explored the potential of our model performance through data selection by eliminating the early COVID period data. After dropping the early COVID period, the results suggest an average of 3.9% increase in AUROC and an average of 1.9% increase in the accuracy. This helps the decision tree model with gradient boosting algorithm achieve 70% in AUROC and 66% in the accuracy.

Various methods in the stock price prediction field can be classified into two categories. The first category aims to improve the performance by developing new structure or combinations of the models. The second class focuses on finding new informative features related to the stock market to help the stock price movement prediction.

In particular, in the first category that focuses the model construction, many different computational models have been used, including Convolutional Neural Network (CNN), Artificial Neural Network (ANN), Long Short Memory Networks (LSTM), Support Vector Machine (SVM), and Random Forests. In 2011, Kara’s team implemented and compared ANN and SVM models performance [8]. They selected ten comprehensive parameters and passed into both models to predict daily stock price movement of Istanbul Stock Exchange National 100 index. The results suggested that ANN models constantly outperform SVM in terms of accuracy in the stock price movement prediction.

Recurrent Neural Network is designed to have internal memories that could help to extract historical characteristics and information. This internal memory could be effective in the stock price prediction domain as stock price database are mostly time oriented and the historical trend information is important. Several research teams have conducted experiments on various Recurrent Neural Networks. In the research from Nelson and his team, the LSTM model was constructed and leveraged with technical indicators as inputs to predict the stock price movement direction of the near future in the Brazilian stock market [9]. The results suggested an average accuracy from LSTM of 55.9%, which outperforms Multilayer Perceptron (MLP) Model. In 2018, Ficher and Krauss applied LSTM with RMSProp as an optimizer to predict stock price movement in S&P 500 from Dec 1989 and Sep 2015. The authors found that the performance from LSTM is better than deep neural network and logistic regression classifier. The extended coverage of the stock market leveraged the function of the long term memory unit in the LSTM. McNally’s research team implemented a combination of LSTM and a Bayesian based RNN to forecast the price of Bitcoin [10]. Their dataset ranges from August 2013 to July 2016. Their results showed a similar performance comparing to random forest classifier.

Persio and Honchar implemented and compared different models including MLP, CNN, and RNN to predict the daily based S&P 500 index [11]. The results showed that the CNN model with wavelet transform outperforms other models with accuracy of 53.6%.

Ramos-Pérez and his team developed a combination of different models to predict S&P 500 volatility [12]. In particular, they stacked Gradient Descent Boosting, Random Forest, SVM, and ANN to make the prediction of S&P 500 and evaluated the performance on an annual basis. The result suggested a better performance from stacked networks than other individual models in term of RMSE.

In addition to the improvement on the models, some researches focus more on the feature selection to improve the model performance. Zhang’s team established an approach to extract user sentiments and web news from social media to improve their stock price movement prediction performance in China A-share and HK stock data in 2015 [13]. They employed a coupled matrix and tensor factorization framework to investigate the impacts of the events (user sentiments and web news) on the stock price movements. Nam and Seong developed Multiple Kernel Learning (MKL) that depends on the financial newsfeed to predict the stock price movement [14]. The MKL model is used to combine the features of target companies and normal companies. They also employed context-aware text mining based on company-specific financial news to gain a better performance. In 2021, Zhang and his team implemented a Convolutional Neural Network model based on a deep factorization machine and attention mechanism (FA-CNN) with different feature engineering techniques to predict the stock price movement [15]. Their results suggested that the inclusion of the industry index information helps to improve the accuracy by 10.2% on average with FA-CNN. Based on the work from Zhang and his team, we also included the industry index as our independent feature along with other COVID-related features.

There are also some data reprocessing techniques that is relevant to our experiment. In the study from Kotsiantis’ team, they explained that the data normalization is important especially when there is a large difference between the minimum and maximum [16]. In particular, they mentioned two normalization techniques: min-max normalization and z-score normalization. In our experiments, we have to collect an extended range of data. Within the same feature, the differences between value two data points could be enormous, such as different stock price value across different companies. In our study, we implemented the min-max normalization to our features.

Based on these previous work, in this study we will compare the performance of three different models: MLP with Logistic regression, Decision Tree with Gradient Boosting, and Decision Tree with Random Forests, with different feature selection combinations, data selection techniques, and online machine learning methods, on 7 selected pharmaceutical stock price over the COVID period.

In this section, we will explain our research design for the stock pricing direction prediction problem, including our methodologies on dataset prepossessing, main and secondary evaluation metrics, and feature engineering.

In this study, we compared two different types of machine learning models: MLP with logistic regression and decision trees. The structure of the MLP model we used is shown in Figure 1. Decision trees are one of the most commonly used models for stock price direction prediction. We also compared boosting and bagging algorithms with random forest: gradient boosting algorithm and random forest bagging algorithm.

To further explore the potential of the dataset, we implemented online machine learning techniques. In our case, we update the model parameters and decision boundaries while evaluating on each test date. In the real world deployment, as the new data points coming, including new daily COVID statistics and stock

price key values, it is important to keep the models being updated to learn the latest connection. The offline learning pattern may hurt the model performance when the new relationships and connections between the features and stock price movements are different from what the models learned using the past dataset. Even though the test dataset is relatively small thus the performance gain from online machine learning techniques may be limited. But we believe this technique will help to keep the model up to date in the real world implementation.

We explored which combination of model and algorithm performs better and results in higher level of AUROC and accuracy. The loss function used by logistic regression and decision tree with gradient descent both are both Binary Cross-Entropy loss. The cross entropy can be use to derive the total entropy between two distributions, which in our case the labels and the predictions. As training process decreases the cross entropy loss, the model learns to make the prediction distribution close to the label distribution.

L = ∑ i = 1 T     y i × log ( p ( y i ) ) + ( 1 − y i ) × log ( 1 − p ( y i ) )

After dividing the dataset into training and testing, we inputted those into these three different computational models. For ablation study purpose, we compared the model performance by removing features I i and C i k individually and together with the model performance with full features we collected. We also compared the model prediction performance on full COVID period (2019.11.30-2021.08.30) with later stage of COVID period (2020.10.30-2021.08.30) to examine if dropping the early stage of COVID periods helps the models to perform better. In the model comparison, we also included the baseline of random guessing, which randomly pick 1 or 0 as the output label.

With the setup above, we conducted several empirical tests. The results of decision tree model with gradient boosting (DCGB) are shown below in Table 2.

The baseline DCGB without COVID and Industry index features gives a 58% accuracy, which is relatively lower accuracy comparing to the an average of 59% accuracy of state-of-art FA-CNN model [15]. But their training data include the period far before COVID. In addition to the smaller dataset, the pandemic introduced more unpredictable factors into the stock market, thus a slight decrease of approximately 1% is within our expectation.

The introduction of handcraft features with outside information (COVID and Industry stock price) helps to increase both the metric dimensions: a 4% increase in AUROC and 5.2% increase in accuracy compared to the baseline performance. Comparing to the state-of-art FA-CNN model [15] with the industry stock price, our results with full feature is 3.2% higher than the average accuracy from FA-CNN model with only industry stock price information.

The model performance improvement from the baseline from COVID related features in both the metric dimensions are greater than that from industry stock price feature: 3% increase in AUROC from COVID handcraft features comparing to 0.4% increase in AUROC from industry stock price features; 4% increase in Accuracy from COVID handcraft features comparing to 2% increase in Accuracy from industry stock price features. Figure 2 shows the AUROC improvement with addition of COVID feature, addition of industry stock price feature, and with full features that include both.

The AUROC and Accuracy are further improved after removing the initial period of COVID. With full features including COVID statistics and Industry stock price statistics, AUROC increases by 5% and the accuracy increases by 2.8%.

The results of Decision Tree model with random forests (DCRF) are shown in Table 3. We only include the accuracy in the table as the AUROC does not apply to the outputs from DCRF. The overall performance of DCRF is relatively lower than that of DCGB, but only with a small difference in terms of accuracy. We explored possible reasons behind the performance difference between DCRF and DCGB in the later section.

For the MLP with logistic regression, the results are shown in Table 4. From the results, the introduction of COVID-related statistics increase both measuring metrics: increase AUROC by 10% and increase accuracy by 9%. The industry stock price statistics improve the metrics performance as well, but with smaller amount. With all of those features, as we expect, the overall performance of the Multi-Layer Perceptron Neural Networks (MLP) obtained a similar improvement. Like the decision tree models, after dropping the early period of COVID, there is a constant enhancement through both measuring metrics. These changes all align with previous models results, even with different absolute values of change.

We also implemented the online machine learning on full COVID period with Full features across all models. The performance gain by adding online machine learning techniques are shown in Table 5. The result suggests an increase of 1.4% in AUROC for DTGB model. The performance gain for the other two models in AUROC and ACC are relatively smaller. We think it is because of our limited testing data. The online machine learning is most effective to improve the model performance when the new coming data has different connections and relationships between the features and labels comparing to the training data. In our dataset, we think the test data is very similar to the training data because they are close in time. However, in the real world application, the online machine learning techniques is necessary as it helps to avoid keeping outdated relationships and connections learned from the old training set.

The best model in this situation is the decision tree with gradient boosting algorithm (DCGB), as DCGB outperforms other models in both AUROC and accuracy consistently in all feature compositions. In the next section, we will dive into the meaning of performance increment difference between COVID statistics and industry stock price information.

Stock price prediction is a comprehensive task. A huge amount of information can affect the stock market. It became more challenging when the pandemic started. The stock market became more unstable and there were lots of subjective feelings, like anxiety and fears, affecting the stock price. Because of the limited length of observation on COVID effects on the stock market and the limited number of pharmaceutical companies conducting COVID vaccines, the scale of our dataset is not very large. However, with the inclusion of COVID-related handcraft features and other information, we are able to increase our model performance comparable to other state-of-art models. We further take a closer look into the performance gain introduced by those outside-of-company handcraft features.

The objective of this paper is to tackle the challenging problem of predicting stock price change direction in the pharmaceutical sub-industry during the COVID period. We explored the performance of different models with different bagging and boosting algorithms. We also examined the performance gain of including a variety of external handcraft features in addition to the conventional internal financial information. These external handcraft features include industry and sub-industry index stock key values and COVID-related statistics.

The result has shown the COVID-related statistics and information, along with related industry index, helps to provide a constant increase in performance across different models in the COVID period. The decision tree with Gradient Boosting algorithm outperforms other models, including MLP with Logistic Regression and decision tree with random forest. The data selection by dropping the first a few months helps the DTGB achieve 70% in AUROC and 66% in the accuracy. The performance gain from online machine learning is less significant. We think it is because of the relatively high similarity between our training and test data sets in the limited time coverage.

In the future, with more data points available, we believe there is a great potential in more complicated machine learning models, specifically LSTM. Given the nature of our dataset (time-series oriented), LSTM could learn from continuous historical trends, instead of dataset rows being learned independently in the current models. As the feature size increases rapidly (189 currently) as we collect more data from different perspectives, feature selection and reduction is another direction that is worth exploring for the next step.

The authors declare no conflicts of interest regarding the publication of this paper.

He, B.L., Han, W.Y. and Hon, S.Y.I. (2022) A Machine Learning Approach: Enhancing the Predictive Performance of Pharmaceutical Stock Price Movement during COVID. Journal of Data Analysis and Information Processing, 10, 1-21. https://doi.org/10.4236/jdaip.2022.101001

Data sources: yfinance (https://pypi.org/project/yfinance/), ycharts, (https://ycharts.com/), and our world in data (https://ourworldindata.org).