Abstract

Forecasting future expected returns out-of-sample is challenging due to some statistical characteristics, such as the stochastic and dynamic nature in the time series. Conventional machine learning techniques focus mainly on point forecasting, which cannot take distributional properties of the returns into account. The Generalised Additive Models for Location, Scale and Shape (GAMLSS) fills this gap by forecasting both the expected returns and the distribution of the returns, thereby making it easier to verify the statistically distributed properties and ensuring models’ validity over the out-of-sample periods. In this paper, we obtained a dataset from the Amit Goyal webpage consisting of 15 financial variables, each covering monthly observations from January 1970 to December 2022. The study has revealed the effectiveness of GAMLSS as a flexible distributional regression in forecasting the U.S. stock market out-of-sample using a rolling window with alternative recursive window methods, to ensure robustness in the analysis. The rolling approach efficiently dealt with any problem of structural dynamics or adverse economic conditions across business cycles. The GAMLSS models demonstrate statistical evidence of superiority over the conventional machine learning techniques across all out-of-sample forecasting windows. Notably, the inclusion of smoothing splines as potential smoothers in the GAMLSS helps to statistically improve the predictive task of the forecasting models. In addition, the GAMLSS models are more economically effective than the buy-and-hold trading strategy, which relies solely on the risk-free treasury bills. Thus, the out-of-sample rolling window forecasts produced by the GAMLSS models tend to be more promising in guaranteeing the fate of investors and portfolio managers while undertaking risky investments with target expectations in a real-time market setting.

Keywords

GAMLSS, Machine Learning, Model Diagnostics, Out-of-Sample, Forecasts

Share and Cite:

1. Introduction

Forecasting the stock market out-of-sample with precision amidst structural dynamics or adverse economic conditions is a research problem associated with financial data. Over the years, traditional time series methods have been used in forecasting future expected returns out-of-sample by rolling or expanding window in previous research but are generally known for producing low accuracy [1]-[3]. Notwithstanding the low forecast accuracies, they seem to be economically meaningful to investors and portfolio decision makers. A notable limitation of these traditional methods is that they focus mainly on point forecast, thereby forecasting the location only, such as the mean, without accounting for the dispersion and shape. In addition, they lack stable diagnostic approaches for visualising or checking the consistency and validity of the resulting models over the out-of-sample periods.

In recent times, conventional machine learning techniques such as elastic net, ridge, least absolute shrinkage and selection operator, generalised additive model, Gaussian process and support vector machines have been employed [4]-[11] in forecasting stock market out-of-sample. These techniques are proven to be effective and superior to traditional time series models, both statistically and economically. As in the case of traditional time series models, these techniques also focus on forecasting a single parameter (mean or volatility) without sufficiently accounting for their distribution over time. Some of these machine learning techniques are tree-based while others rely on shrinkage penalty vector norms that penalise or regularise the model to increase bias in coefficient estimates but minimise prediction error, and hence the bias-variance trade-off. However, there are approaches for detecting underfitting and overfitting when training and validating the models, but they lack a stable long-run distributional quality and diagnostic testing for checking consistency and validity over time. This existing gap triggers the introduction of flexible distributional regression with diagnostic testing approaches when forecasting the stock market.

In this paper, we proposed the Generalised Additive Models for Location, Scale and Shape (GAMLSS) as a flexible distributional regression with smoothing splines in forecasting stock market out-of-sample by rolling window. The GAMLSS was introduced by [12] as a semi-parametric statistical model that uses a distributional approach to model the parametric distribution of the response variable thereby allowing the location, scale and shape to depend on the explanatory variables [13]. GAMLSS is a supervised machine learning technique in machine learning framework. Hopefully, the GAMLSS models will fill the gaps, owing to their distributional properties and provision of stable diagnostic approaches for visualising or checking the consistency and validity of the resulting models over the out-of-sample periods. The application of rolling window method together with inclusion of smoothing splines and diagnostic checks in the GAMLSS models will adequately model the data amidst structural dynamics or adverse economic conditions across business cycles and produce more accurate out-of-sample forecasts in this direction.

The rest of the paper is organised as follows: Section 2 provides the literature review, Section 3 describes the research methodology, Section 4 gives the results and discussion, and Section 5 concludes the paper and provides an identifiable area for future research.

2. Literature Review

A major challenge in the analysis of financial time series data is the determination of suitable explanatory variables and choice of statistical models that could adequately fit the data and consistently produce accurate out-of-sample forecasts, due to their complicated features. In particular, the choice of predictors and modelling techniques that could be statistically powerful in forecasting future expected market returns is underway. Some scholars have argued that financial and macroeconomic variables are more useful in forecasting stock returns [1] [14]-[16]. [1] and [17] added that short interest rate is the strongest known predictor of aggregate stock returns, while others are useful at short horizons. Others have argued that technical indicators are more useful in forecasting future expected returns than macroeconomic indicators [18]-[20]. [18] suggested that a combination of both macroeconomic and technical indicators significantly improves forecast accuracy using either of the two alone. However, these analyses are mainly based on U.S. information and are subject to further investigation with other international markets.

Some well-known models are statistically powerful in forecasting stock market returns in-sample but do not significantly yield out-of-sample economic gains. This led to the introduction of the historical average as a benchmark by [21]. It implies that any competing model that could not significantly outperform the historical average out-of-sample by rolling or recursive window does not possess more statistical and economic power than the historical average. Previous studies have shown that other modelling strategies such as model restrictions, forecast combination, diffusion indices and regime shifts are superior to the benchmark historical average, both statistically and economically [22]. [2] demonstrated the potential of autoregressive integrated moving average (ARIMA) models in forecasting time series stocks using Netflix stock historical data for five years. However, this argument lacks comparative justification as it requires further investigation possibly by comparing the analysis with the benchmark relative returns from S&P500.

Some scholars in previous studies have argued that machine learning techniques are more instrumental in forecasting stock market than basic statistical and financial time series models. Another critical assertion in this regard is whether the machine learning techniques are more effective in forecasting the direction of the return or the return itself. Evidently, both approaches are proven to be effective as shown in empirical literature [4] [23]-[26]. The penalised binary probit models introduced by [4] significantly outperformed the static and dynamic binary probit models used by [14]. Supervised learning is predominantly used in machine learning techniques in stock market forecasting, as it involves the use of labelled data sets or instances to train algorithms that could recognise patterns and predict future outcomes. As machine learning encourages inclusion of many features as covariates, the algorithms could use computational methods to learn information directly from the time series data without relying on a predetermined model. [9] and [27] have shown that the presence of penalty vector norms in the ridge, LASSO and elastic nets together with their tuning parameters play active roles in regularising and improving the predictive tasks of these models, with evidence of superiority over the conventional financial time series models. [28] have confirmed that machine learning models are more effective at discerning valuable prediction of cross-sectional stock returns than traditional linear models.

A recent study by [29], which investigates the content of volatility indices, has further confirmed that machine learning models outperformed the classical least squares linear regression when forecasting the direction of S&P500 returns. [30] argued that an artificial neural network is a superior machine learning algorithm to logistic regression, decision tree, random forest, k-nearest neighbour, naive Bayes and support vector machine algorithms in forecasting the directional movement of stocks, using indexes from other developed countries. In their study, they sought indices from the USA (NYSE 100), Japan (NIKKEI 225), the UK (FTSE 100), France (CAC 40), Germany (DAX 30), Italy (FTSE MIB), and Canada (TSX). Unlike the traditional financial models that rely mainly on the U.S. information, the effectiveness of machine learning techniques is not limited to the U.S. information alone, as they have consistently shown evidence of effectiveness in the stock market returns of many developed countries other than the USA.

Notwithstanding the superiority of these machine learning models, there exists a gap in distributional properties and stable diagnostic approaches for consistently validating the forecasting model and guaranteeing accurate forecasts over the out-of-sample periods. Also, the mean squared forecast errors could be minimised by employing further approaches that could adequately fit the data. [31] have attempted to predict the distribution of stock returns but could only predict the density under a combination scheme when applied to univariate generalised autoregressive conditional heteroscedastic (GARCH) models. Thus, the GAMLSS models introduced by [12], [13] and [32] as flexible semi-parametric distributional regression with distributional properties such as the location (mean), scale (variance) and shape (skewness and kurtosis) and smoothing splines are proposed to fill these research gaps. As the GAMLSS models with smoothing splines are aimed to align with rolling approach as well as the diagnostic checks, we opined that the GAMLSS models will adequately fit the data amidst structural dynamics or adverse economic conditions such as recession or COVID-19 and yield consistent forecast accuracies better than the conventional machine learning techniques used in previous studies. In addition, GAMLSS models can accommodate many features and can display a weak correlation between the response variable and the explanatory terms, thereby addressing multicollinearity and overfitting to ensure robustness in the analysis.

3. Methodology

As stated in the literature, there exist some statistical characteristics, especially the stochastic and dynamic nature in the time series of expected returns, that make it difficult to accurately forecast future returns. This is because the distribution of the returns could change over time, which might influence the out-of-sample forecasts and affect investment outcomes. Therefore, it would be helpful to begin by modelling the distribution of the expected returns and proceed thereafter with the distributional regression.

3.1. Distribution of Expected Returns

Let $Y_{t+1}$ represent a random vector of stock index based on S&P500 with $y_{t+1}$ representing the index in the new month $t+1$ , and $y_t$ representing the index in the old month $t$ . The uncertainty of the stock index from months $t$ to $t+1$ will lead to a probability distribution for $Y_{t+1}$ in the form:

$Y_{t+1}~y_{t|t+1}\left(r_{t+1},{\Omega }_{t+1}\right)$

where $r_{t+1}$ is the vector of expected returns; and ${\Omega }_{t+1}$ is the positive semi-definite covariance.

For a mean-variance portfolio optimisation, we assume that the portfolio returns are normally distributed. Let $R_{t+1}$ denote the portfolio returns, relating to $y_{t|t+1}$ , then $R_{t+1}$ conditional on the information set (covariates) $X_t$ follow normal distribution:

$R_{t+1}|X_t~N\left({\mu }_{t+1},{\sigma }_{t+1}^2\right)$

with mean ${\mu }_{t+1}=w_{t|t+1}^{\text{T}}{r}_{t+1}$ and variance ${\sigma }_{t+1}^2=w_{t|t+1}^{\text{T}}{\Omega }_{t+1}{w}_{t|t+1}$ .

3.2. Generalised Additive Model

The generalised additive model (GAM) introduced by [33] is an extension of the general linear model by replacing the linear terms with flexible smoothing functions of explanatory variables, which could be non-parametric, semi-parametric or pre-specified parametric form. Let (X, Y) be n-pairs of random variables with realisations defined by ${\left\{X_i,Y_i\right\}}_{i=1}^n$ , where Y is the response variable and X is k-dimensional vector of explanatory variables. The generalised additive model will take the form:

$g\left({\mu }_i\right)=\eta \left(X_i\right)={\beta }_0+f_1\left(X_{i1}\right)+f_2\left(X_{i2}\right)+\cdots +f_k\left(X_{ik}\right)+{ϵ}_i$ (1)

where $\mu =E\left[Y|X\right]$ is the conditional mean; $Y_i\text{'}s$ are independent and identically distributed with mean ${\mu }_i$ and variance ${\delta }^2$ ; ${\beta }_0$ is the intercept; $f_1\left(X_{i1}\right),f_2\left(X_{i2}\right),\cdots ,f_k\left(X_{ik}\right)$ are additive predictors representing the relationship between the k-dimensional covariates and the response variable Y via the link function $g(.)$ ; ${ϵ}_i$ is the error term.

Let $s_j\left(X_j\right)$ represent smooth functions of covariates for $j=1,2,\cdots ,k$ . Then model (1) in matrix notation will be:

$g\left(\mu \right)=\eta \left(X\right)=X\beta +{\displaystyle {\sum }_{j=1}^k{s}_j\left(X_j\right)}$ (2)

where $X\in {ℝ}^{n\times \left(k+1\right)}$ is design matrix; and $\beta ={\left({\beta }_0,{\beta }_1,{\beta }_2,\cdots ,{\beta }_k\right)}^{\text{T}}$ is a vector of unknown parameters.

3.3. Generalised Additive Models for Location, Scale and Shape

Let ${\left\{y_i\right\}}_{i=1}^n$ denote independent observations of a response variable $Y$ with probability density function $f_Y\left(y_i|{\mu }_i,{\sigma }_i,{\nu }_i,{\tau }_i\right)$ conditional on the four distribution parameters, $y_i~D\left({\mu }_i,{\sigma }_i,{\nu }_i,{\tau }_i\right)$ , with each parameter serving as a function of the explanatory variables.

Let $g_k(.),k=1,2,3,4$ be known monotonic link functions connecting the distribution parameters to explanatory variables by:

$g_k\left({\theta }_k\right)={\eta }_k=X_k{\beta }_k+{\displaystyle {\sum }_{j=1}^{J_k}{Z}_{jk}{\gamma }_{jk }}$ (3)

and the four parameters are individually modelled as follows:

$g_1\left(\mu \right)={\eta }_1=X_1{\beta }_1+{\displaystyle {\sum }_{j=1}^{J_1}{Z}_{j1}{\gamma }_{j1}}$ (4)

$g_2\left(\sigma \right)={\eta }_2=X_2{\beta }_2+{\displaystyle {\sum }_{j=1}^{J_2}{Z}_{j2}{\gamma }_{j2}}$ (5)

$g_3\left(\upsilon \right)={\eta }_3=X_3{\beta }_3+{\displaystyle {\sum }_{j=1}^{J_3}{Z}_{j3}{\gamma }_{j3}}$ (6)

$g_4\left(\tau \right)={\eta }_4=X_4{\beta }_4+{\displaystyle {\sum }_{j=1}^{J_4}{Z}_{j4}{\gamma }_{j4}}$ (7)

where $\mu ,\sigma ,\nu ,\tau$ and ${\eta }_4$ are vectors of length $n$ ; $\mu ,\sigma ,\nu$ & $\tau$ are respectively known as the location, scale and shape (skewness and kurtosis) parameters; ${\beta }_k^{\text{T}}=\left({\beta }_{1k},{\beta }_{2k},\cdots ,{\beta }_{{J^{\prime }}_{k}k}\right)$ is a parameter vector of length ${J^{\prime }}_k$ , $X_k$ is a fixed known design matrix of order $n\times {J^{\prime }}_k$ ; $Z_{jk}$ is a fixed known $n\times q_{jk}$ design matrix and ${\gamma }_{jk}$ is a $q_{jk}$ dimensional random variable which is assumed to be distributed as ${\gamma }_{jk}~N_q{}_{jk}\left(0,G_{jk}^{-1}\right)$ , where $G_{jk}^{-1}$ is the generalised inverse of a $q_{jk}\times q_{jk}$ symmetric matrix $G_{jk}=G_{jk}\left({\lambda }_{jk}\right)$ which may depend on a vector of hyperparameters ${\lambda }_{jk}$ ; and if $G_{jk}$ is singular then ${\gamma }_{jk}$ is understood to have an improper prior density function proportional to $\exp \left(-\frac{1}{2}{\gamma }_{jk}^{\text{T}}{G}_{jk}{\gamma }_{jk}\right)$ [12] [32].

The flexibility of GAMLSS makes it possible to include smoothing splines or additive functions such as penalised beta splines ${\Psi }_{pb}$ , penalised varying coefficients function ${\Psi }_{pvc}$ and cubic splines ${\Psi }_{cs}$ , respectively in model (3), as follows:

${\Psi }_{pb}\left\{g_k\left({\theta }_k\right)\right\}={\Psi }_{pb}\left\{X_k{\beta }_k+{\displaystyle {\sum }_{j=1}^{J_k}{Z}_{jk}{\gamma }_{jk}}\right\}$ (8)

${\Psi }_{pvc}\left\{g_k\left({\theta }_k\right)\right\}={\Psi }_{pvc}\left\{X_k{\beta }_k+{\displaystyle {\sum }_{j=1}^{J_k}{Z}_{jk}{\gamma }_{jk}}\right\}$ (9)

${\Psi }_{cs}\left\{g_k\left({\theta }_k\right)\right\}={\Psi }_{cs}\left\{X_k{\beta }_k+{\displaystyle {\sum }_{j=1}^{J_k}{Z}_{jk}{\gamma }_{jk}}\right\}$ (10)

where $pb(.)$ , $pvc(.)$ and $cs(.)$ are smoothers influencing the explanatory variables in the GAMLSS models.

The parametric vectors ${\beta }_k$ and the random effects parameters ${\gamma }_{jk}$ , for $j=1,2,\cdots ,J_k$ and $k=1,2,3,4$ are estimated within the GAMLSS framework (for fixed values of the smoothing hyper-parameters ${\lambda }_{jk}$ ) by maximising a penalised likelihood function ${\ell }_p$ given by

${\ell }_p=\ell -\frac{1}{2}{\displaystyle {\sum }_{k=1}^p{\displaystyle {\sum }_{j=1}^{J_k}{\lambda }_{jk}{{\gamma }^{\prime }}_{jk}{G}_{jk}{\gamma }_{jk}}}$ (11)

where $\ell ={\displaystyle {\sum }_{i=1}^n\log f\left(y_i|{\theta }^i\right)}$ is the log likelihood function for $j=1,2,\cdots ,J_k$ and $k=1,2,3,4$ .

To compare the out-of-sample rolling window forecasts obtained by the GAMLSS models with the conventional machine learning techniques, we can extract the expected return forecasts produced by the location parameter (mean), while setting the scale (variance) and shape (skewness and kurtosis) as null in the gamlss package in R. We use the continuous response distribution across all GAMLSS models, owing to the continuous nature of the stock market return variable.

3.4. The Conventional Machine Learning Models

We employ some conventional machine learning models consistently used in previous literature for the purpose of replicating the analyses and comparing the outputs with the results of GAMLSS models. The conventional machine learning models and their architectures with indicative sources for further reading are the Gaussian Process Regression [34]-[37], Elastic Net [9] [38], Ridge Regression [39] [40], Least Absolute Shrinkage and Selection Operator [41] [42], Artificial Neural Network [30] [43]-[45], and Support Vector Machines [46]-[49].

3.5. Performance Evaluation

To assess the effectiveness of the GAMLSS models and the out-of-sample rolling window forecasts in comparison to the conventional machine learning techniques, we employ suitable statistical and economic performance evaluation metrics. These metrics include Mean Squared Forecast Error (MSFE), Out-of-Sample R-Squared (${\text{R}}_{\text{OOS}}^2$ ), Diebold-Mariano (DM) Test, Maximum Drawdown (maxDD), Upside Potential Ratio (UPR), Sortino Ratio (SortR), Sharpe Ratio and Utility Gain, respectively. These metrics are consistently used across all windows to ensure robustness in the analysis. The Utility Gain is based on the approach used in [9] and [21]. We use the three-month U.S. Treasury bill (T-bill) yield as a proxy for the risk-free rate series.

4. Results and Discussion

The dataset used in this paper is obtained from the Amit Goyal webpage, consisting of 15 financial variables, each covering monthly observations from January 1970 to December 2022. These financial variables are described in Table 1. Most of these variables have been consistently used in previous studies and are proven to be useful in forecasting future expected stock returns. In order to efficiently handle the problem of structural changes or market dynamics across business cycles, we employ the rolling window method for training the models and running the forecasts out-of-sample. The results were further confirmed by an expanding or recursive window method to ensure robustness in the analysis of this paper. The rolling method makes use of a fixed window of data to re-estimate the parameters and update the models over the out-of-sample periods, whereas the expanding method makes use of an increasing window to re-estimate the parameters of the models recursively over the out-of-sample periods. In-sample training window lengths consist of 383 observations for Window 1, 467 observations for Window 2, and 563 for Window 3. The out-of-sample periods are split into three distinct windows, namely Window 1 covering forecasts from January 2002 to December 2008, Window 2 covering forecasts from January 2009 to December 2016, and Window 3 covering forecasts from January 2017 to December 2022, respectively. The optimal out-of-sample window lengths used in this study are 84 for Window 1, 96 for Window 2, and 72 for Window 3. In pre-processing the training dataset, we performed data cleaning to handle missing values and check for potential outliers for each in-sample window. We employ the resampling approach involving centring and scaling as pre-processing steps in fine-tuning each model parameter, to determine the parameters that produce the least mean square forecast error (MSFE) in each case. The lag length chosen by Akaike information criterion (AIC) for all explanatory variables is one. The centring and scaling are introduced to improve each model’s performance, speed up the training process and ensure equal treatment of features in the machine learning framework.

Table 1. Description of variables used in the study.

Four suitable GAMLSS models were investigated, namely GAMLSS 1, GAMLSS 2, GAMLSS 3, and GAMLSS 4. GAMLSS 1 does not contain any smoothing splines, GAMLSS 2 contains penalised beta splines as a smoothing additive function, GAMLSS 3 contains penalised varying coefficients function, and GAMLSS 4 contains cubic smoothing splines. The GAMLSS models are compared with other recently and consistently used class of machine learning techniques in stock market predictability. These techniques are Elastic Net, Generalised Additive Model (GAM), Gaussian Process—GP (with linear, radial basis and polynomial kernel functions), Least Absolute Shrinkage and Selection Operator (LASSO), Artificial Neural Network (ANN), Ridge, Support Vector Machines—SVM (with linear, radial basis and polynomial kernel functions), respectively. We employ appropriate statistical and economic performance evaluation metrics to assess the performance of each forecasting model.

Following the benchmark suggested by [9] [21] and [50] in which any competing out-of-sample forecasting model is to be compared with the historical average, we compare the performance of each model with the historical average in the first case. In the out-of-sample window 1, the GAMLSS models and all the other machine learning techniques consistently beat the historical average (${\text{R}}_{\text{OOS}}^2>0$ ). Also, the GAMLSS models adequately fit the data, better than the conventional machine learning techniques in window 1, as evidenced by their respective out-of-sample R-squared values. The GAMLSS models generally produced smaller mean squared forecast errors (MSFEs) than the other machine learning techniques used in the study (see Table 2). GAMLSS 4 gives the smallest MSFE, followed by GAMLSS 3, GAMLSS 2, and GAMLSS 1. It could entail evidently that the inclusion of smoothing splines or smoothing additive terms in the GAMLSS helps to statistically improve the predictive task of the model. Using GAMLSS 1 as a benchmark, the Diebold-Mariano (DM) tests of equal versus unequal forecast accuracies provide strong statistically significant evidence that GAMLSS 2, GAMLSS 3 and GAMLSS 4 have unequal forecast accuracies with GAMLSS 1. Also, there is strong statistically significant evidence that GAMLSS 1 does not have equal forecast accuracy with the Elastic Net, GP (Linear, Radial, Poly), LASSO, Ridge and SVM (Linear, Radial, Poly) respectively. However, GAMLSS 1 does not give any significant evidence of unequal forecast accuracy with GAM and ANN, respectively.

Table 2. Forecast evaluation for out-of-sample window 1: January 2002 to December 2008.

Note: MSFE = mean squared forecast error, ${\text{R}}_{\text{OOS}}^2$ = out-of-sample R-squared, ${\text{DM}}_{\text{p value}}$ = Diebold-Mariano p-value, maxDD = maximum drawdown, UPR = upside potential ratio, SortR = Sortino ratio. GAM = Generalised Additive Models, GP = Gaussian Process (with linear, radial basis and polynomial kernel functions respectively), LASSO = Least Absolute Shrinkage and Selection Operator, SVM = Support Vector Machines (with linear, radial basis and polynomial kernel functions respectively), GAMLSS = Generalised Additive Models for Location Scale and Shape; GAMLSS 1 = GAMLSS without smoothing splines, GAMLSS 2 = GAMLSS with penalised beta splines, GAMLSS 3 = GAMLSS with penalised varying coefficients function, GAMLSS 4 = GAMLSS with cubic smoothing splines. The risk aversion parameter = 3.

In the economic evaluation metrics, the GAMLSS models generally produced results that are economically meaningful to investors and portfolio managers. [4] and [6] have stated in their studies that the superiority of a forecasting model in terms of statistical performance evaluation does not necessarily imply superiority in economic significance. In agreement with this claim, the GAMLSS models do not correspondingly yield the best maximum drawdown (maxDD), upside potential ratio (UPR), Sortino ratio (SortR) and Sharpe ratio, even though they have outperformed most of the other machine learning techniques. Notwithstanding, the GAMLSS models seem to suggest low investment losses, the potential for investment to increase in value and better risk management, such that an investment could be efficient at managing losses and severity of risk associated with the investment, according to the analysis (see Table 2). Unlike the Sharpe ratio, the Sortino ratio only penalises returns with downside risks and provide mean-variance investors with the level of risk they could face over time. Interestingly, the GAMLSS models generally produced higher utility gains than the other machine learning techniques, with GAMLSS 4 yielding the best economic output among the other GAMLSS in this direction. The utility gain is economically meaningful as it provides the portfolio management fee that an investor could be willing to pay in order to access the additional information in the out-of-sample forecasting model relative to the sole information in the historical returns. In this regard, the numerical value of the utility gain in each model is compared with the average risk-free or treasury bill rate, which serves as a benchmark for investors to decide whether to invest in the risky portfolio or to stick to the buy-and-hold trading strategy. It is worth noting in window 1 that all the GAMLSS models give utility gains greater than the average treasury bill rate, thereby guaranteeing investors’ fate on risky portfolio investment than the buy-and-hold trading strategy.

Unlike the other machine learning techniques used in the study, the GAMLSS models have a more stable statistical and graphical (visualisation) approach for checking and ensuring validity of the models, especially the distribution over the out-of-sample periods. All the GAMLSS models in window 1 clearly indicate satisfactory assumptions as judged by the normal density curves, Q-Q plots, and worm plots, respectively (see Figure 1(a) and Figure 1(b)). The worm plots provide confirmative evidence about the normality in that the residuals follow normal distribution, with no significant deviation or outlier. Model selection by Akaike information criterion (AIC) was investigated and GAMLSS 2 emerged as the best model with the lowest AIC value. The normality and worm plots for GAMLSS 2 are depicted in Figure 1(a) and Figure 1(b) respectively, clearly aiding visualisation in terms of diagnostics. Also, the GAMLSS models as distributional regression have the flexibility of statistically forecasting both the monthly expected returns (location) together with other parametric measures of the returns, including the variance (scale), skewness and kurtosis (shape), thereby enriching empirical literature. In each case, the distribution of the mean (location) is approximately zero and the variance (scale) is approximately one (see Table 5), which statistically confirms the distributional assumptions produced by the diagnostic plots. The coefficients of skewness and kurtosis fall within the normal and acceptable range, clearly indicating further evidence of normality. The Filliben correlation coefficient which serves as probability plot correlation coefficient indicates that the shape parameter of the distribution best describes the set of data. Thus, the provision of these flexible statistics contributes immensely to ensuring robustness in the analysis.

In the out-of-sample window 2, all the GAMLSS models and the other machine learning techniques consistently beat the historical average (${\text{R}}_{\text{OOS}}^2>0$ , and lower MSFEs), with the GAMLSS models outperforming the other machine learning techniques, in terms of statistical predictability. Again, GAMLSS 4 yields the lowest MSFE, followed by GAMLSS 3, GAMLSS 2, and GAMLSS 1 (see Table 3). The DM test provides statistically significant evidence that GAMLSS 4 and GAMLSS 1 have unequal forecast accuracies. Also, the analysis revealed that GAMLSS 1 has unequal forecast accuracies with Elastic Net, GP (Linear, Radial, Poly), LASSO, ANN, Ridge, and SVM (Linear, Radial, Poly) respectively. It could imply that GAMLSS 1 is more statistically powerful in predictive power than these techniques. However, there is no significant evidence that GAMLSS 1 has unequal forecast accuracy with GAM, GAMLSS 2 and GAMLSS 3, respectively. Thus, the presence of cubic smoothing splines in the GAMLSS 4 seems to be more statistically effective in terms of predictive power than GAMLSS 2 and GAMLSS 3. All the GAMLSS models adequately fit the data, better than the conventional machine learning techniques in window 2, as evidenced by their respective out-of-sample R-squared values.

(a)

(b)

Figure 1. (a) Diagnostic (normality) plots showing the distribution of residuals for GAMLSS 2; (b) Diagnostic (worm) plot showing the distribution of residuals for GAMLSS 2.

Table 3. Forecast evaluation for out-of-sample window 2: January 2009 to December 2016.

Note: MSFE = mean squared forecast error, ${\text{R}}_{\text{OOS}}^2$ = out-of-sample R-squared, ${\text{DM}}_{\text{p value}}$ = Diebold-Mariano p-value, maxDD = maximum drawdown, UPR = upside potential ratio, SortR = Sortino ratio. GAM = Generalised Additive Models, GP = Gaussian Process (with linear, radial basis and polynomial kernel functions respectively), LASSO = Least Absolute Shrinkage and Selection Operator, SVM = Support Vector Machines (with linear, radial basis and polynomial kernel functions respectively), GAMLSS = Generalised Additive Models for Location Scale and Shape; GAMLSS 1 = GAMLSS without smoothing splines, GAMLSS 2 = GAMLSS with penalised beta splines, GAMLSS 3 = GAMLSS with penalised varying coefficients function, GAMLSS 4 = GAMLSS with cubic smoothing splines. The risk aversion parameter = 3.

Again, all the GAMLSS models clearly indicate evidence of satisfying the normality assumptions, both statistically and graphically (see Table 5, Figure 2(a), Figure 2(b)), with no indicative evidence of deviation. The fact that the residuals are normally distributed with zero mean and variance one, together with the flexible inclusion of other useful distributional statistics, makes the GAMLSS models uniquely different from other machine learning techniques. In terms of model selection by AIC and summary of quantile residuals, GAMLSS 2 gives the best results. In the economic performance evaluation, the GAMLSS models generally provide useful results that could potentially guide mean-variance investors and portfolio managers in optimal decision making amidst uncertainty. Though the GAMLSS 4 gives the best statistically predictive results but does not correspondingly yield the best economic results. Instead, it is economically suggesting the highest investment loss as indicated by the maxDD, with corresponding lowest Sharpe ratio and utility gain among the other GAMLSS models. Thus, it justified the claim that the statistical superiority of a forecasting model does not necessarily imply corresponding superiority in economic significance.

Turning to the out-of-sample window 3, all the GAMLSS models and the other machine learning techniques consistently beat the historical average, with the GAMLSS models consistently outperforming the other machine learning techniques, statistically (see Table 4). As in windows 1 and 2, the GAMLSS 4 gives the lowest MSFE, followed by GAMLSS 3, GAMLSS 2, and GAMLSS 1. The DM test provides statistically significant evidence that GAMLSS 1 does not have the same forecast accuracy as the other GAMLSS models. Also, it is worth noting that GAMLSS 1 is significantly different in forecast accuracy when compared with each of the other machine learning techniques used in the study. Again, the emergence of GAMLSS 4 as the best statistically performing GAMLSS across all forecasting windows is indicative evidence of consistency in superiority, amidst structural changes or adverse market conditions across business cycles. Also, the cubic smoothing splines in the GAMLSS 4 appeared to be potentially more useful in statistically predictive power than the penalised beta splines and penalised varying coefficients function, as potential smoothers.

(a)

(b)

Figure 2. (a) Diagnostic (normality) plots showing the distribution of residuals for GAMLSS 2; (b) Diagnostic (worm) plot showing the distribution of residuals for GAMLSS 2.

Table 4. Forecast evaluation for out-of-sample window 3: January 2017 to December 2022.

Note: MSFE = mean squared forecast error, ${\text{R}}_{\text{OOS}}^2$ = out-of-sample R-squared, ${\text{DM}}_{\text{p value}}$ = Diebold-Mariano p-value, maxDD = maximum drawdown, UPR = upside potential ratio, SortR = Sortino ratio. GAM = Generalised Additive Models, GP = Gaussian Process (with linear, radial basis and polynomial kernel functions respectively), LASSO = Least Absolute Shrinkage and Selection Operator, SVM = Support Vector Machines (with linear, radial basis and polynomial kernel functions respectively), GAMLSS = Generalised Additive Models for Location Scale and Shape; GAMLSS 1 = GAMLSS without smoothing splines, GAMLSS 2 = GAMLSS with penalised beta splines, GAMLSS 3 = GAMLSS with penalised varying coefficients function, GAMLSS 4 = GAMLSS with cubic smoothing splines. The risk aversion parameter = 3.

The diagnostic plots with tests in the out-of-sample window 3 also revealed that all the GAMLSS models clearly indicate evidence of satisfying the normality assumptions (see Table 5, Figure 3(a), Figure 3(b)), with no evidence of deviation or outlier. Contrary to windows 1 and 2, the model selection by AIC and summary of quantile residuals revealed that GAMLSS 3 is the best among the other GAMLSS models tested. It could be perceived that the presence of penalised varying coefficients function as smoothers in the GAMLSS 3 seemed to be more potentially adaptive to structural break, market dynamics or adverse conditions such as COVID-19 pandemic in the out-of-sample window 3. In terms of statistical goodness of fit, the GAMLSS models adequately fit the data across all out-of-sample windows compared to the conventional machine learning techniques. Turning to the economic performance metrics, not all the GAMLSS models correspondingly outperformed the other machine learning techniques. Notwithstanding, the GAMLSS models generally provide meaningful economic information that could potentially support mean-variance investors and portfolio managers in optimal decision making amidst uncertainty. Unlike in window 2, the GAMLSS 4 correspondingly gives the highest utility gain in window 3.

(a)

(b)

Figure 3. (a) Diagnostic (normality) plots showing the distribution of residuals for GAMLSS 3; (b) Diagnostic (worm) plot showing the distribution of residuals for GAMLSS 3.

Table 5. Model selection & summary of quantile residuals across all estimation windows.

The smoothing splines in the GAMLSS models seemed to align with rolling approach, as evidenced in the statistics and diagnostic checks, thereby making the GAMLSS models adequately fit the data amidst structural dynamics or adverse economic conditions and yielding consistent forecast accuracies across all out-of-sample windows better than the conventional machine learning techniques used in previous studies. Model selection by AIC revealed that GAMLSS 2 best fits the data in estimation windows 1 and 2 respectively, while GAMLSS 3 best fits the data in estimation window 3. Like the conventional machine learning techniques, the GAMLSS models allow many features as predictors, displaying a weak correlation between the response variable and the predictors, evidently addressing multicollinearity and overfitting to ensure robustness in the analysis. GAMLSS 1 uses fewer parameters than the other GAMLSS models. GAMLSS 2 and GAMLSS 3 used approximately the same number across all estimation windows, while GAMLSS 4 uses more parameters than other GAMLSS models, as indicated in their respective degrees of freedom (df). It is noticeable that the forecast accuracies for GAMLSS 2 and GAMLSS 3 are approximately the same across all out-of-sample windows if the MSFEs are rounded to four decimal places. This could imply that the penalised beta splines and the penalised varying coefficients function tend to use the same number of parameters and possess approximately the same statistically predictive power in stock return out-of-sample predictability. Whereas the cubic splines in GAMLSS 4 which employed more parameters significantly produced the highest forecast accuracies across all out-of-sample forecasting windows.

Evidently, the GAMLSS models demonstrate statistical evidence of superiority over the conventional machine learning techniques across all out-of-sample forecasting windows (see Tables 2-4). Notably, the inclusion of smoothing splines or smoothing additive functions in the GAMLSS helps to statistically improve the predictive task of the forecasting model. In addition to the statistical superiority, the GAMLSS models are more economically effective than the buy-and-hold trading strategy which relies solely on the risk-free treasury bills in a real-time market setting. Thus, the out-of-sample rolling window forecasts produced by the GAMLSS models tend to be more promising in guaranteeing the fate of investors and portfolio managers while undertaking risky investments with a target expectation.

5. Conclusions

The study has revealed the effectiveness of GAMLSS as a flexible distributional regression in forecasting the U.S. stock market out-of-sample using a rolling window with alternative recursive window methods. In the rolling window method, the problem of structural changes or market dynamics across business cycles was efficiently handled over the out-of-sample periods. We investigate four GAMLSS models, which include GAMLSS 1, GAMLSS 2, GAMLSS 3, and GAMLSS 4. GAMLSS 1 does not contain any smoothing additive function terms, GAMLSS 2 contains penalised beta splines as smoothing additive terms, GAMLSS 3 contains penalised varying coefficients function as smoothing additive terms, and GAMLSS 4 contains cubic smoothing splines. The results of these GAMLSS models are compared with conventional machine learning techniques recently used in forecasting expected stock return out-of-sample.

In this paper, the GAMLSS models are uniquely different from the conventional machine learning techniques, as they forecast both the future expected stock returns and the distributional properties of the returns across all out-of-sample forecasting windows. Unlike other techniques that focus on mean or point forecasting alone, the GAMLSS models can forecast the location (mean or expected return), together with the scale (variance) and shape (skewness, kurtosis). In terms of model selection by AIC, GAMLSS 2 gives the best result in both out-of-sample windows 1 and 2, respectively, whereas GAMLSS 3 is rated as the best in window 3. In terms of statistical goodness of fit, the GAMLSS models adequately fit the data across all out-of-sample windows compared to the conventional machine learning techniques. The summary of quantile residuals and diagnostic plots was investigated to ensure satisfactory assumptions and validity of each GAMLSS model. The diagnostic plots with tests in all out-of-sample revealed that all the GAMLSS models indicate evidence of satisfying the normality assumptions. At each stage, the residuals are distributed with approximately zero mean and variance one, with approximate evidence of normality as depicted in the normal density curves and Q-Q plots, with no evidence of deviation or outlier. Thus, the distributional quality of the GAMLSS models together with the rolling window method ensures consistency, stability and robustness amidst structural changes or adverse market conditions such as COVID-19 across business cycles.

Interestingly, all the GAMLSS models consistently outperformed the conventional machine learning across all out-of-sample forecasting windows, statistically. It is worth noting that the inclusion of smoothing splines or smoothing additive functions in the GAMLSS helps to statistically improve the predictive task of the forecasting model. Among the GAMLSS models tested, GAMLSS 4 gives the smallest mean squared forecast error (MSFE) and is regarded as the best in terms of statistical predictive power. Thus, the cubic smoothing splines in the GAMLSS 4 appeared to be potentially more useful in statistically predictive power than the penalised beta splines and penalised varying coefficients function, as potential smoothers. Turning to the economic performance metrics, the GAMLSS models correspondingly outperformed the conventional machine learning techniques economically in window 1, but do not economically outperform in windows 2 and 3. This corroborates the analyses in previous findings, which affirms that the statistical superiority of a forecasting model does not necessarily imply corresponding superiority in economic significance. Notwithstanding, the GAMLSS models generally provide meaningful economic information that could potentially support mean-variance investors and portfolio managers in optimal decision-making amidst uncertainty.

Overall, the GAMLSS models demonstrate statistical evidence of superiority over the conventional machine learning techniques across all out-of-sample forecasting windows. Additionally, the GAMLSS models are more economically effective than the buy-and-hold trading strategy, which relies solely on the risk-free treasury bills in a real-time market setting. Future researchers can explore the topic further by transforming the expected returns in a closed interval and investigating the feasibility of Beta-inflated and Gamma-inflated distributions within the GAMLSS framework.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References