This paper investigates the tolerable sample size needed for Ordinary
Least Square (OLS) Estimator to be used when there is presence of Multicollinearity
among the exogenous variables of a linear regression model. A regression model
with constant term (β0)
and two independent variables (with β1 and β2 as their respective
regression coefficients) that exhibit multicollinearity was considered. A Monte
Carlo study of 1000 trials was conducted at eight levels of multicollinearity
(0, 0.25, 0.5, 0.7, 0.75, 0.8, 0.9 and 0.99) and sample sizes (10, 20, 40, 80,
100, 150, 250 and 500). At each specification, the true regression coefficients
were set at unity while 1.5, 2.0 and 2.5 were taken as the hypothesized value.
The power value rate was obtained at every multicollinearity level for the
aforementioned sample sizes. Therefore, whether the hypothesized values highly
depart from the true values or not once the multicollinearity level is very
high (i.e. 0.99), the sample size
needed to work with in order to have an error free estimation or the inference
result must be greater than five hundred.
Regression Model OLS Estimator Multicollinearity Power Rate Value and Tolerable Sample Size1. Introduction
There has been a serious argument between the researchers that multicollinearity problem could be solved with the increase of the sample size while some researchers say that Multicollinearity problem will also increase with the increase in the size of the sample. [1] stated that Multicollinearity problem could be solved by increase of the size of the sample if the presence of multicollinearity is due to errors of measurement as well as when intercorre- lation happens to exist only in our original sample but not in the population [2] . Because of these arguments this paper then investigates the tolerable sample size needed for Ordinary Least Square Estimator to be used when there is presence of Multicolinearity among the exogenous variables of a linear regression model before we can say that multicollinearity problem could be solved with increase of the sample size method.
Regression theory postulates that there exists a stochastic relationship between a variable and a set of other variables. In other words, (called the dependent, endogenous or explained variable) depends on other observed variables, (called independent, exogenous or explanatory variables). However, one of the assumptions of this model is that the explanatory variables are independent. This is not often the case in economic variables. Variables like age and year of experience do exhibit a form of linear relation- ship. When this assumption is violated, it results into multicollinearity problem [3] .
Multicollinearity could be perfect or imperfect. When it is perfect, estimates obtained are not unique [4] . If multicollinearity is not perfect, the OLS estimator has been shown to be unbiased but inefficient. Other consequences or indications of multicollinearity problem include:
1. Small changes in the data can produce significant changes in the parameter estimates (regression coefficients).
2. The regression coefficients may have wrong signs and/or unreasonable magnitudes.
3. Regression coefficients have high standard errors which result in very low values of the t-statistic and thus affect the significance of the parameters [3] [5] .
Thus, the presence of multicollinearity in a data set does not only affect parameter estimation using the OLS estimator but also inferences on the parameters of the model. Consequently, with generated collinear data, this paper attempts to investigate empirically the most tolerable sample size where power rate value of 0.99 or 1 would be obtained with ordinary least square (OLS) estimator.
2. Methodology
Consider the regression model of the form
where
is the dependent variable,
and are regressors which exhibit correlation (multicollinearity), and, , and are the regression coefficient (parameters) of the model.
Now, suppose. If these variables are correlated, then and can be generat- ed with the equations
where and is the value of correlation between the two variables [6] ; and [7] .
Monte Carlo experiments were performed 1000 times for eight sample sizes (n = 10, 20, 40, 80, 100, 150, 250 and 500) and eight levels of multicollinearity (ρ = 0, 0.25, 0.5, 0.7, 0.75, 0.8, 0.9 and 0.99) with stochastic regressors that are normally distributed. At a particular specification of n and (ascenario), the first replication was obtained by generating. Next, and were generated using Equation (2) such that they exhibit correlation. The values in Equation (1) were obtained by taking the true regression coefficients as unity. This process is continued until all the 1000 replications had been done. Another scenario is then started until all the scenarios were completed. For each replication in the scenario, the OLS estimator of parameter estimation was used to obtain estimate of the regression coefficients and hypothesis about the true regression coefficient was tested at 0.05 level of significance using the t-statistic to examine the type II error of the regression coefficients. All these were done by writing a computer program using the Time Series Processor (TSP) software. The result of the effect of type II error rate on OLS estimators by [8] was considered by taken the type II error rate away from 1 to obtain the power rate value for every sample sizes at all levels of multicollinearity. These power rate values were then considered at all levels of multicollinearity for all the selected sample sizes. Then the sample size with the power rate value of 0.999 or 1.0 was chosen as the most tolerable sample size at each level of multicollinearity and different parameter values, [9] on effects of multicollinearity on the power rates of the Ordinary least Squares Estimators.
3. Results and Discussion
The summary of the most tolerable sample sizes at different level of multicolinearity and different possible com- bination of the parameter values are shown for, and in Tables 1-8.
When the true values of and are maintained and that of is allowed to change, The summary of the tolerable sample sizes required for the parameter to have a power rate value of 0.99 or 1 was determin- ed at different levels of multicollinearity and hypothesized values. The results for these are shown in Table 1.
Table 1. The tolerable sample sizes for when the true values of and are maintained and that of are chan- ging at different levels of multicollinearity.
Table 1
Table 1. The tolerable sample sizes for when the true values of and are maintained and that of are chan- ging at different levels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
1.5, 1, 1
250
250
250
250
250
250
250
250
2, 1, 1
80
40
40
40
40
40
40
40
2.5, 1, 1
20
20
20
20
40
40
40
40
Table 2. The tolerable sample sizes for when the true values of and are maintained and that of is allowed to change at different levels of multicollinearity.
Table 2
Table 2. The tolerable sample sizes for when the true values of and are maintained and that of is allowed to change at different levels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
1, 1.5, 1
150
150
150
250
250
250
250
>500
1, 2, 1
40
40
80
80
80
100
250
>500
1, 2.5, 1
40
40
40
40
40
40
40
>500
Table 3. The tolerable sample sizes for when the true values of and are maintained and that of is allow- ed to change, at different levels of multicollinearity.
Table 3
Table 3. The tolerable sample sizes for when the true values of and are maintained and that of is allow- ed to change, at different levels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
1, 1, 1.5
150
150
250
250
250
250
250
500
1, 1, 2
80
40
40
40
40
80
80
>500
1, 1, 2.5
20
20
20
20
40
80
80
500
Table 4. The tolerable sample sizes for when the true value for is maintained and that of and are allowed to change at different levels of multicollinearity.
Table 4
Table 4. The tolerable sample sizes for when the true value for is maintained and that of and are allowed to change at different levels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
1, 1.5, 2
150
150
150
250
250
250
500
>500
1, 1.5, 2.5
150
150
150
250
250
250
500
>500
1, 2, 1.5,
40
40
40
80
80
80
100
>500
1, 2, 2.5
40
40
40
80
80
80
100
>500
1, 2.5, 1.5
40
40
40
80
80
80
100
>500
1, 2.5, 2
40
40
40
80
80
80
100
>500
Table 5. The tolerable sample sizes for when true value of is maintained and that of and are allow to change at different levels of multicollinearity.
Table 5
Table 5. The tolerable sample sizes for when true value of is maintained and that of and are allow to change at different levels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
1, 1.5, 2
150
150
150
250
250
250
500
>500
1, 1.5, 2.5
150
150
150
250
250
250
500
>500
1, 2, 1.5,
40
40
40
80
80
80
100
>500
1, 2, 2.5
40
40
40
80
80
80
100
>500
1, 2.5, 1.5
40
40
40
80
80
80
100
>500
1, 2.5, 2
40
40
40
80
80
80
100
>500
Table 6. The tolerable sample sizes for when all the values for, and are allowed to change at different le- vels of multicollinearity.
Table 6
Table 6. The tolerable sample sizes for when all the values for, and are allowed to change at different le- vels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
1.5, 2.5, 2
100
100
100
100
100
100
100
100
2, 1.5, 2.5
40
40
40
40
40
40
40
40
2.5, 2, 1.5
20
20
20
20
20
20
20
20
Table 7. The tolerable sample sizes for when all the values for, and are allowed to change at different le- vels of multicollinearity.
Table 7
Table 7. The tolerable sample sizes for when all the values for, and are allowed to change at different le- vels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
2, 1.5, 2.5
100
100
100
150
250
250
500
500
2.5, 2, 1.5
40
40
40
40
40
40
40
500
1.5, 2.5, 2
40
40
40
40
40
40
80
80
Table 8. The tolerable sample sizes for when all the values for, and are allowed to change at different le- vels of multicollinearity.
Table 8
Table 8. The tolerable sample sizes for when all the values for, and are allowed to change at different le- vels of multicollinearity.
Parameter values
0
0.25
0.5
0.7
0.75
0.8
0.9
0.99
2.5, 2, 1.5
150
150
250
250
250
250
500
500
1.5, 2.5, 2
40
40
40
40
40
80
80
>500
2, 1.5,2.5
40
40
80
80
80
150
250
500
Likewise, when the true values of and are maintained and that of is allowed to change, The summary of the tolerable sample sizes required for the parameter to have a power rate value of 0.99 or 1 was determined at different levels of multicollinearity and hypothesized values. The results for these are shown in Table 2.
When the true values of and are maintained and that of is allowed to change, The summary of the tolerable sample sizes required for the parameter to have a power rate value of 0.99 or 1 was determin- ed at different levels of multicollinearity and hypothesized values. The results for these are shown in Table 3.
The summary of the tolerable sample sizes at different levels of multicollinearity and hypothesized values are shown in Table 3.
Also, for all other possible combinations of the parameter values similar results were obtained.
From Table 1 to Table 8 the tolerable sample size value decreases as the hypothesized values departed from the true values in all lower levels of multicolinearity, whereas at higher levels of multicolinearity the required Tolerable sample sizes increases as the hypothesized values departed from the true value. But at very high level of multicolinearity (0.99) the Tolerable sample size needed must be greater than 500 before a result with.
4. Conclusion
In conclusion, at every multicollinearity level the most tolerable sample size was then obtained as the one with the highest value of power rate, which we were able to obtain at a sample size equal or greater than five hundred. This study has revealed that whether the hypothesized values were highly depart from the true values or not once the multicolinearity level is very high (i.e. 0.99), and the sample size needed to work with in other to have an error free estimation or inference result must be greater than five hundred, if and only if, increments of the size of the sample method would be used as a measure of correction to the presence of multicollinearity.
ReferencesSTONE, R. (1961) THE MEASUREMENTS OF CONSUMER EXPENDITURE AND BEHAVIOR IN UNITED KINGDOM. CAMBRIDGE PUBLISHING COMPANY.KOUTSOYIANNIS, A. (2003) THEORY OF ECONOMETRICS. 2ND EDITION, PALGRAVE.CHARTERJEE, S., HADI, A.S. AND PRICE, B. (2000) REGRESSION ANALYSIS BY EXAMPLE. 3RD EDITION, WILEY-INTERSCIENCE PUBLICATION, JOHN WILEY AND SONS.SEARLE, S.R. (1971) LINEAR MODELS. JOHN WILLEY AND SONS, NEW YORK.FOMBY, T.B., HILL, R.C. AND JOHNSON, S.R. (1984) ADVANCED ECONOMETRIC METHODS. SPRINGER-VERLAG, NEW YORK, BERLIN, HEIDELBERG, LONDON, PARIS, TOKYO.AYINDE K. ,et al. (2006)A COMPARATIVE STUDY OF THE PERFORMANCES OF OLS AND SOME GLS ESTIMATOR WHEN REGRESSORS ARE BOTH STOCHASTIC AND COLLINEAR. WEST AFRICAN JOURNAL OF BIOPHYSICS AND BIOMATHEMATICS 2, 54-67.AYINDE K. , OYEJOLA B.A. ,et al. (2007)AYINDE, K. AND OYEJOLA, B.A. A COMPARATIVE STUDY OF THE PERFORMANCES OF OLS AND SOME GLS ESTIMATOR WHEN STOCHASTIC REGRESSORS ARE CORRELATED WITH ERROR TERMS. RESEARCH JOURNAL OF APPLIED SCIENCES 2, 215-220.ALABI, O.O. (2007) EFFECTS OF MULTICOLINEARITY ON TYPE 1 AND TYPE 11 ERRORS OF ORDINARY LEAST SQUARES ESTIMATORS. UNPUBLISHED M.SC. THESIS SUBMITTED TO THE DEPARTMENT OF STATISTICS UNIVERSITY OF ILORIN, ILORIN.ALABI O.O., AYINDE K. , OLATAYO T.O. ,et al. (2008)ON EFFECTS OF MULTICOLLINEARITY ON THE POWER RATES OF THE ORDINARY LEAST SQUARES ESTIMATORS. JOURNAL OF MATHEMATICS AND STATISTICS 4, 75-80.HTTP://DX.DOI.ORG/10.3844/JMSSP.2008.75.80
when the true values of 
and 
are maintained and that of 
are chan- ging at different levels of multicollinearity.
Parameter values
when the true values of 
and 
are maintained and that of 
is allowed to change at different levels of multicollinearity.
Parameter values
when the true values of 
and 
are maintained and that of 
is allow- ed to change, at different levels of multicollinearity.
Parameter values
when the true value for 
is maintained and that of 
and 
are allowed to change at different levels of multicollinearity.
Parameter values
when true value of is maintained and that of 
and 
are allow to change at different levels of multicollinearity.
Parameter values
when all the values for
, 
and 
are allowed to change at different le- vels of multicollinearity.
Parameter values
when all the values for
, 
and 
are allowed to change at different le- vels of multicollinearity.
Parameter values
when all the values for
, 
and 
are allowed to change at different le- vels of multicollinearity.
Parameter values