When dealing with validation of data, usually there exists a portion not explained by the model, which combines two parts, the whiteness and the independence test. According to the first part (whiteness test criteria), a good model has the residual autocorrelation function within the confidence interval of the corresponding estimates, indicating that the residuals are uncorrelated; as according to the second criteria of independence test, residuals of a good model should be uncorrelated with previous elements. In fact, there is a need to check residuals in order to validate model performance in regression analysis, which is the main purpose of this empirical study.

Several procedures are used to analyze data within these domains. A useful common technique is the Box-Jenkins ARIMA method [1] , which can be used for univariate or multivariate data set analyses. The ARIMA technique involves using Moving Averages (MA), smoothing and regression techniques to detect and manipulate data autocorrelation problems. Error modeling approach has clearly been demonstrated by Firmino et al. [2] and Ikughur, A. J., et al. [3] .

Diagnostic investigating and checking ARIMA time series models and residual analysis techniques have been used by McLeod and Li [4] . Lu [5] has introduced forecasting for China GDP of a time series model. Andreii and Bugudui [6] have presented an econometric modeling of GDP time series in the US economy. Other similar forecasting models using residual analysis were found in Okyere et al. [7] , Boshnakov [8] and Lavrenz et al. [9] . Martin et al. [10] showed checking regression assumptions (errors normality, constant variance, residual analysis plots) in regression analysis fitting experimental data via residual plots.

Residuals Analysis

Consider the regression model:

y = a + b x + e (1)

The gap between the original value of the dependent variable (y) and the estimated value (ŷ) is known as the residual (e). It is the amount of variability in dependent variable that is (left over) after accounting for the variation explained by the predictors in regression analysis. Each data point has one residual:

e = y − y ^ ,

where Σe = 0 and e = 0.

Seen as a powerful diagnostic tool, checking the residuals of a regression is a way of checking whether a regression has achieved its goal to explain as much variation as possible in a dependent variable.

Ideally all residuals should be small and unstructured. Most problems that were initially overlooked when diagnosing the variables in the model or were impossible to see, will, turn up in the residuals, for instance:

• Big residuals shown due to outliers

• Certain structure will appear in residuals, will appear in the nonlinearity of the residuals.

• Heteroscedasticity problem, a type of violation of the assumption of non-constant model variation of the residuals.

• Examining residuals plots to check appropriateness of regression model for the data [11] .

• If the points in a residual plot indicates appropriateness of a linear or non-linear regression model for the data, Lavrenz et al. [9] .

The residuals plot shows a certain pattern. Random pattern might be an indication of goodness of fit to the data of a linear model. A form of an autoregressive process time series is as:

x t = b 0 + b 1 x t − 1 + ∈ t (2)

For the basic analysis of residuals you will use the usual descriptive tools and scatter plots. A Q-Q Plot can be used to test for residual normality, besides Plotting the residuals to see if there appears any particular pattern (random cloud). A researcher may need to decide whether to adopt linear or log-linear trend models after answering some questions related to the estimated relationship around the trend line and the correlation of the error terms.

The Autoregressive Time Series Models

Abbreviated as AR(p) models, where p stands for the number of the lagged values of the dependent variable which known as the model “order”. The “order” of the AR(p) models is the number of prior values used in the model. Thus:

AR ( 1 ) : x t = b 0 + b 1 x t − 1 + ∈ t (3)

AR ( 2 ) : x t = b 0 + b 1 x t − 1 + b 2 x t − 2 + ∈ t (4)

and so on.

When considering the autocorrelation of the residuals that are to be used to evaluate model fit, the testing procedure includes estimating the AR model and calculating of residuals (or error terms), estimating the autocorrelations for the error terms (residuals) and testing to see the inside structure of autocorrelations to see if statistically different from zero. For an AR(1), the values will stay constant when:

x t = b 0 / ( 1 − b 1 )

And rise when:

x t < b 0 / ( 1 − b 1 )

The value falls if:

x t > b 0 / ( 1 − b 1 )

To determine whether a time series is an AR(p) or a Moving average-MA(q)-, examine the autocorrelations. Generally, AR model autocorrelations start at large values and then decline gradually, while the MA autocorrelations drop dramatically after reach of q lags. This behavior describes both the MA process and its order.

Types of Residual Plot

In first graph bellow, linear regression model is preferred since dots are randomly dispread, while in second and third plot, a non-linear regression method is suggested since dots are non-randomly dispread. The randomness property in the first plot shows indicates a good fit for a linear model, as the rest of the plot patterns indicate a non-random structure which expressed by (U and inverted U shaped), suggesting a non-linear model structure (Figure 1).

Expected Mean Square (EMS)

Expected Mean Square (EMS) represent the values that we will get for any given mean square (MS) statistic under distribution, on average over repeated experiment.

Let:

μ ¯ ∑ i = 1 k μ i / k λ i = μ i − μ ¯ ,     and σ A 2 = ∑ i = 1 k λ i 2 , ( Itisvariancelike )

Let σ e 2 be the true error variance. Then the ANOVA is as Table 1 and Table 2.

The Proposed Model

Let Y be the target predicting value and Y_t is the value of Y at time t, then we aimed at constructing a model of the type:

Y t = f ( Y t − 1 , Y t − 2 , Y t − 3 , ⋯ , Y t − n ) + e t (5)

where, Y_t−1 is the previous observation value of Y, Y_t−2 is the value two observations before, etc. and e_t (a random shock or the noise term). The values of underlying variables that occur prior to the current observation are called lag values. In a repeating pattern time series, the value of Y_t is expected to be highly correlated with Y_t−cycle. Thus, the goal of constructing a time series model is to build a model such that the error term to be as small as possible.

Let us consider an X_t, time series model, then the Autoregressive Moving Average (ARMA) model combines two components, an autoregressive (AR) component and a moving average (MA) part. Following Brockwell and Davis [12] and Pasavento [13] , the AR(p) model can be written in the form:

Y t = c + ∑ i = 1 p φ i Y t − i + ε t (6)

where, φ 1 , ⋯ , φ p are the model parameters, c is a constant (which may be omitted for simplicity) and e_t is an error term. The MA(q) notation stands for the moving average model of order q:

Y t = ε t + ∑ i = 1 q   θ i ε t − i (7)

where, the θ 1 , ⋯ , θ q are the parameters of the model and the ε t , ε t − 1 , ⋯ is an error term.

The ARMA(p,q) notation represents the p autoregressive terms and q moving average terms of the underlying model. This model is given the notation AR(p) and MA(q) models, which is:

Y t = ε t + ∑   i = 1 p φ i   Y t − i + ∑ i = 1 q     θ i   ε t − i (7)

where the error terms ε_t are distributed randomly and assumed to be independent with mean zero and variance σ² that is ε t ~ N ( 0 , σ 2 ) .

The process (Y)_t is defined to be ARIMA(p, d, q) if:

( 1 − l ) d ∅ * ( l ) Y t = c + θ ( l ) ε t (9)

where, ∅ * ( l ) is defined in ∅ ( l ) = ( 1 − l ) ∅ * ( l ) , ∅ * ( z ) ≠ 0 for all | z | ≤ 1 . And q(l) is defined in θ ( z ) ≠ 0 for all | z | ≤ 1 .

The process (Y)_t is said to be stationary (i.e., its mean, variance and autocorrelation should be approximately constant through time) if and only if d = 0 in which case it eliminates to ARMA(p, q) process:

∅ ( l ) Y t = c + θ ( l ) ε t (10)

where, ε t ~ w N ( 0 , σ 2 ) .

The Box-Jenkins methodology Box et al. [1] is a five-step procedure for analyzing and assessing time series data models. Adopting the ARIMA (auto-regressive, integrated, moving average) method iteratively, to best-fit time series data, then the (AR) component in ARIMA is structured as p, the integrated component (I) as d and moving average (MA) as q. The (AR) component represents the previous observations effects, where the (I) component represents model trends and the MA part holds effects of previous random shocks (or error). Then the order of the time series ARIMA fit can be selected assigning an integer value (0, 1, or 2) for each component.

The Sudan Central Bureau of Statistics (CBS), issue annual report include all National accounts, while the Central Bank of the Sudan [14] also issues its annual economic records. United Nations forms are used for annual gatherings of the official national accounts data presented to the United Nations Statistics Division by the countries, according to International Monetory fund [15] . If a full set of official data is not reported, estimation methods are used to obtain estimates for the entire time series. Then annual percentage growth indicators including annual rates of GDP based on market prices is presented on constant local currency, while the aggregates are based on constant U.S. dollars [16] .

In this section, the data of GDP statistics of Sudan, which include the current and constant prices in million US$ for the period (1960-2015) will be investigated using SPSS Time Series Modeler. The created series for the GDP model with creating function has been made and presented in Table 3 and result variables are displayed in Table 4. The result variables for the GDP model functions are identical. As in Table 5, applying the model specifications, it seems that it is not applicable for calculating standard errors of the partial autocorrelations, for independence (white noise).

The partial autocorrelation for the built model is the autocorrelation of time series observations separated by a lag of 16 time units with the effects of the intervening observations eliminated. Autocorrelation presented in Table 6 with autocorrelation function (Figure 2)―values seen above zero in (Figure 4) too―and partial autocorrelation tables (Table 7) with partial autocorrelation (Figure 3) are also provided for the residuals (errors) between the actual and predicted values of the time series. Examining the autocorrelation table shown in Table 6, we see that the highest autocorrelation is 0.875 (the first value in the

Applying the model specifications from MOD_1. ^aNot applicable for calculating the standard errors of the partial autocorrelations.

^aThe underlying process assumed is independence (white noise). ^bBased on the asymptotic chi-square approximation.

lags) which occurs with a lag of 15. Now we aim to be sure to include lag values up to 15 when constructing the model. Based on the assumption that the series are not cross correlated and that one of the series is white noise, the cross correlations are found in Table 8.

Using Time Series Modeler, the model specification was shown with range of lags from −7 to +7 as seen in Table 9 and described in Table 10 that is (ARIMA(0,1,0) and the model fit summary is shown in Table 11 and the residuals ACF summary is presented in Table 12.

a. Based on the assumption that the series are not cross correlated and that one of the series is white noise.

Applying the model specifications from MOD_2.

The autocorrelation ACF and partial autocorrelation PACF tables provide valuable information about the significance of the lag variables. An autocorrelation is the correlation between the target variable (GDP) and lag values for the same variable. It is known that correlation values lies between −1 and +1. A value of +1 indicates that the two variables move together perfectly; a value of −1 indicates that they move in opposite directions (see the results of Table 14. The third column of the autocorrelation table shows the standard error of the autocorrelation, this is followed by Box-ljung statistics based on the asymptotic chi-square approximation (all values are significant) in the fourth column. The autocorrelation bar chart is used to indicate positive or negative correlations up or down of the centerline. The dots shown in the chart mark the points two standard deviations from zero. If the autocorrelation bar is longer than the dot marker (that is, it covers it), then the autocorrelation should be considered significant. In this model, significant autocorrelations occurred for all lags except for lag 15. On the basis of the assumption that the series are not cross correlated and that one of the series is white noise, the cross correlations and range of lags (from −7 to +7 are displayed in Table 13 and Figure 4. The figure shows confidence limit to be all above zero for the GDP.

Proportion of variance explained by model is the best single measure of how well the estimated values match the original values. If the estimated values exactly match the original values, then the model would explain all the amount of variation (100%). In fact this is not always the case (here the model explains 98.2% of the variance due to the R square value), as seen in Table 13. The ARIMA model parameters using natural log show significant t value (0.001) as in Table 14. The residual ACF is displayed in Table 15 and the residual PACF is presented in Table 16.

Finally, the model forecast has been shown in Table 17, since one of the strengths of time series model analysis is its ability of generation of future data forecasts by depending on observed past observations. Thus, if we rely on this information, we may conclude that we have a good model fit.

For each model, forecasts start after the last non-missing in the range of the requested estimation period, and end the last period for which non-missing values of all the predictors are available or at the end date of the request forecast period, whichever is earlier.

A residual is the vertical distance between a data point and the regression line. Considering residual plots, it can be used to assess whether the observed error (residuals) is consistent with random (or stochastic) error. In a regression model, the residuals should not be either systematically high or low. So, they should be spread about zero throughout the range of fitted values. Further, in the context of OLS method, random errors are assumed to yield normally distributed residuals. This means that the residuals should be distributed in a symmetrical pattern and spread constantly throughout the range.

We evaluate Autoregressive Integrated Moving Average (ARIMA) model of the GDP series using Box-Jenkins methodology by using four different equations which are, linear, logarithmic, quadratic and exponential equations. Rely on the parameter estimates, it is found that the ARIMA (0,1,0) is the best model for the data. Comparing with other models, ARIMA model has been selected as the final model. The method for prediction and forecasting then has been provided based on data, which may be performable and useful to governmental and nongovernment institutions.

In time series regression analysis of data, caution must be taken about using this technique, because of autocorrelation and violation of the assumption of errors independence which leads to increase in type I error when autocorrelation is present. Furthermore, time series patterns must be accounted for within the analysis.

This article has discussed the technique of residual analysis for an economic GDP model. The procedures―which mainly depend on time series analysis and ARIMA method in particular―used here might be valuable only for a time series that is stationery and it is much preferred and more recommended when carried out for a model with at least 50 data observations (our model has 57 observations). The procedure of residual analysis in an economic time series model is outlined. The model has been investigated to describe the data to see how well the underlying model fits the GDP data which can help econometricians to understand the behavior and structure of the Sudanese economy. Thus, this analysis may provide practical utility of the procedure of residual analysis.

I am deeply indebted to the editorial board and the reviewers of the Journal of Mathematics and Statistics for their valuable comments and efforts in order to publish this paper.

The author declares no conflicts of interest regarding the publication of this paper.

Elsayir, H.A. (2019) Residual Analysis for Auto-Correlated Econometric Model. Open Journal of Statistics, 9, 48-61. https://doi.org/10.4236/ojs.2019.91005