A matrix multiple linear regression model can be written as

where y is an n × 1 response vector, X is an n × p centered and standardized known matrix (i.e. the length of the column of X is standardized to one), is a p × 1 vector of an unknown parameter, e is an n × 1 error vector with and and is an identity matrix of order n. Then the ordinary least squares estimator (OLSE) of is The estimator of is where residual vector

This article assumes that the reader is acquainted with the basic ideas of leverage and influence in regression analysis, as presented, for instance, in the works of [1] and [9].

The general purpose of influential analysis is to measure the changes induced in a given aspect of the analysis when the data set is perturbed. A particularly appealing perturbation scheme is case deletion. Note that this scheme is used throughout this article.

In general, the influence of a case can be viewed as the product of two factors: the first a function of the residual and the second a function of the position of the point in the X space. The position or leverage of the i-th point is measured by h_i, which is the i-th diagonal element of the “hat” matrix.

Among the most popular single-case influential measure is the difference in fit standardized DFFITS [1], which evaluated at the i-th case is given by

where is the least squares estimator of without the i-th case and is an estimator of the standard error (SE) of the fitted values. DFFITS is the standardized change in the fitted value of a case when it is deleted. Thus it can be considered a measure of influence on individual fitted values.

Another useful measure of influence is Cook’s D [9], which evaluated at the i-th case is given by

where is a measure of the change in all of the fitted values when a case is deleted. Even though is based on different theoretical consideration, it is closely related to DFFITS.

Points with large values of have considerable influence on the least squares estimate. In general, points for which to be influential. In DFFITS measures any observation for which

warrants attention.

It is important to mention that these measures are useful for detecting single cases with an unduly high influence. For generalizations of Equations (2) and (3) for detecting influential sets (see [1,9]).

The Liu estimator was introduced by Liu [10] and is defined as

where I is an identity matrix, is OLSE and d is Liu estimator biasing parameter and it is. The Liu estimator combines the ORRE [11,12] estimator. The ORRE is effective in practice, but it is complicated function of its biasing parameter. Thus we often meet some complicated equations when we use some popular methods, such as ([13], C_k criterion [14], GCV criterion [15]) and etc. to choose ridge regression biasing parameter k. The advantage of Liu estimator over ORRE is that Liu estimator is a linear function of its biasing parameter d. Therefore, it is convenient to choose Liu estimator biasing parameter d.

The Liu estimator is very useful to mitigate the effect of near multicollinearity. Also, the recent literature, particularly in the area of econometrics, engineering and other statistical areas, the Liu estimator has produced a number of new techniques and ideas for example [5-7, 16-20].

Using Equation (4), the vector of fitted value of Liu estimator is

where is Liu estimator “hat matrix” (see [2,10]) and plays the same role as the hat matrix in OLSE. It is important to note that the matrix H_d is not a projection matrix because it is not idempotent and H_d is called quasi-projection matrix (see [2]). The i-th fitted value can be written in terms of elements of H_d as consequently,

The “Liu hat diagonals” h_di can be interpreted as leverage in the same sense as the hat diagonals in OLSE. It is important to note that the H_d is not idempotent and it is called a quasi-projection matrix (see [2]).

The single value decomposition (SVD) (see [21]) allows X to be decomposed as X = UDV', where D is a p ´

p diagonal matrix with i-th diagonal elements (

is the i-th eigenvalue of X'X), the column of V are the eigenvectors of X'X. The (ij)-th element of the n ´ p matrix is such that is the projection of the i-th row, x_i, onto the j-th principal axis (eigenvector) of X. Using the SVD, the Liu estimator leverage of the i-th point can be written as (see [22]).

Several important facts can be deduced from the preceding expression. First, for d > 0, for; that is, for every observation the Liu estimator leverage is smaller than the corresponding OLSE leverage. It can be confirmed from the above equation that

Second, the leverage decreases monotonically as d increases. Finally, the rate of decrement of leverage depends on the position of the particular row of X along the principal axes. More specifically, the leverage of a row that lies in the direction of principal axes associated with large eigenvalues will be reduced less than the leverage of a row that lies in the direction of principal axes associated with small eigenvalues (see [2]).

The influence can be differentially affected as d increases. Remember, that influence is not a function of leverage but also of the residual. Although the leverage of every point decreases monotonically as d increases, the effect of this increment on the residuals is far less clear.

The i-th Liu estimator residual is defined as

The DFFITS for Liu estimator can be written as

where is the Liu estimator in (4) computed with the i-th case deleted and the denominator is an estimator of the standard error of the Liu estimator fitted value. If Liu estimator biasing parameter d is assumed non-stochastic, then

Hence the mean squared error is a function of the fitted values and the response, neither of which depends on individual eigenvalues of it is not affected by multicollinearity. For this reason, the OLSE of will be used as measures of scale.

At least two versions of Cook’s D_i can be constructed for Liu estimator, they are

and

where is the direct generalization of Cook’s D in (3) and is based on the fact that

Note that both and simplify to D_i in (3) when d = 1.

It would be desirable to be able to write these measures as functions of leverage and residual, as was done in (2) and (3). This is not possible, however, because of the scale dependency of the Liu estimator. Since the Liu estimator is not scale invariant, (the X matrix with the i-th row deleted) has to be rescaled to unit-column length before computing In the following section some approximate deletion formulas are proposed.

Sun [23] studied MIFDI in China 1979-1996. Based on his theory, the MIFDI data were collected in Sri Lanka form 1978 to 2004 to illustrate the methodologies proposed in this paper. The data set consists of four regressors (Foreign Direct Investment, Gross Domestic Product Per Capita, Exchange Rate and Interest Rate) and one response variable (Total Domestic Investment) with 27 observations. The selected variables were tested for statistical conditions: Integration and Multicollinearity. The test results showed that variables are integrated with a same order of integration I(1) at 1% level of significance. The scaled condition number of this data set is 31,244, this large value suggests the presence of an unusually high level of multicollinearity among the regressors (the proposed cutoff is 30; see [1]). The Liu estimator biasing parameter for this data set is estimated d = 0.692.

Global influence measures Leverage, Residual, DFFITS and two versions of Cook’s D_i were computed and the results for the most seven influential cases are given in Table 1. The influential cases detected by these methods are same except case 26 in leverage and case 14 in residual measures, but only the order of magnitude is changed.

Using the approximate case deletion formulas in Equations (8)-(10) the influential cases are estimated and it is given in Table 2. From this table, it can be seen that the influential cases detected by approximate case deletion formulas DFFITS and two versions of Cook’s D_i are same but only the order of magnitude is changed. Moreover, influential cases detected by case deletion one by

Table 1. The most seven influential observations using leverage, residual, DFFITS and two versions of Cook’s.

one formulas in Equations (5)-(7) and approximate deletion formulas in (8)-(10) are exactly same but only the order of magnitude is changed.

For verifying these results, it is plane to contribute Table 3 of Liu estimates for the full data and the data without some influential cases. In this table, the parenthesis value indicates the percentage of change in the parameter value. The result reveals that case 3 is the most influential case while case 22 is the seventh influential point among the detected cases. It is also clear from this table, that omission of single influential cases 3, 23, 1, 27, 2, 18 and 22 contribute the substantial change in the Liu estimator. Among all of these 3, 23, 1, 2 and 18 have a remarkable influence while cases 27 and 22 have a little influence.

The Longley data set [8] has also been used to explain the effect of extreme multicollinearity on the OLSE. The scaled condition number (see [1]) of this data set is 43,275. This large value suggests the presence of high level of multicollinearity among regressors. Cook [24] used this data to identify the influential observations in OLSE using the method of Cook’s D_i and found that cases 5, 16, 4, 10 and 15 (in this order) were the most influential observations (see Table 4). Walker and Birch [2] analyzed the same data to detect anomalous observations in ORRE using case deletion method. They found that cases 16, 10, 4, 15 and 5 (in this order) were most influential observations (see Table 4). Shi and Wang [25] also analyzed the same data to detect influential observations on the ridge regression estimator using local influence method. They detected cases 10, 4, 15, 16 and 1 (in this order) were the most five anomalous observations.

In this paper, I used the same data set to assess the influential observations in Liu estimator using global influence methods: Cook’s D_i, DFFITS, Leverage and Residual. The estimated results, the most five influential cases and the corresponding values are given in Table 4.

The approximate deletion formulas in Equations (8)- (10) are used to detect influential cases for Longley data and detected influential cases are given in Table 5. According to this table, it can be confirmed that identified influential cases using approximate deletion formulas DFFITS and two versions of Cook’s D_i are exactly same. Besides, influential cases identified by case deletion one by one formulas in Equations (5)-(7) and approximate deletion formulas in (8)-(10) are precisely same for Longly data.

The influential cases in MIFDI and Longly data were identified using one by one deletion formulas in Equations (5)-(7) and approximate deletion formulas in Equations (8)-(10) respectively. The identified influential cases for MIFDI and Longly data are same for both measures but only the order of magnitude is changed for MIFDI data. Hence, instead of using the one by one case deletion formulas to identify influential cases in Liu estimator the approximate case deletion formulas are more suitable and appropriate to identify influential cases in Liu estimator.

Table 2. The most seven influence observations according to approximate case deletion formulas.

Table 3. Impact of influential cases on Liu estimator parameter.

Table 4. The five most influence observations in Longley data.

Table 5. The most five influential cases for Longly data using deletion formulas.