TITLE:
New Facts in Regression Estimation under Conditions of Multicollinearity
AUTHORS:
Anatoly Gordinsky
KEYWORDS:
Linear Regression, Multicollinearity, Two Classes of Regression Models, Shrinkage Estimators, Inequality Constrained Least Squres Estimator, Dual Estimator
JOURNAL NAME:
Open Journal of Statistics,
Vol.6 No.5,
October
21,
2016
ABSTRACT: This paper considers the approaches and
methods for reducing the influence of multi-collinearity. Great attention is
paid to the question of using shrinkage estimators for this purpose. Two
classes of regression models are investigated, the first of which corresponds
to systems with a negative feedback, while the second class presents systems
without the feedback. In the first case the use of shrinkage estimators,
especially the Principal Component estimator, is inappropriate but is possible
in the second case with the right choice of the regularization parameter or of
the number of principal components included in the regression model. This fact
is substantiated by the study of the distribution of the random variable ,
where b is the LS estimate and β is the true coefficient, since the form of this
distribution is the basic characteristic of the specified classes. For this
study, a regression approximation of the distribution of the event based on the
Edgeworth series was developed. Also, alternative approaches are examined to
resolve the multicollinearity issue, including an application of the known
Inequality Constrained Least Squares method and the Dual estimator method
proposed by the author. It is shown that with a priori information the
Euclidean distance between the estimates and the true coefficients can be
significantly reduced.