A detailed presentation of Variance and Covariance Components Models can be found in Rao and Kleffe [1], as well as in Krishnaiah [2].

However, in the applications deviations appear from the assumptions under which these solutions are obtained, which also leads to negative estimates of variance components. The main source of these deviations is the contaminating distribution of observations (also presence of outliers), which, as a rule, always appears. Therefore, Variance Components (VC) solutions are always studied, which are less sensitive or insensitive to these deviations. Such properties are characteristic for robust solutions, out of which the following ones are here presented:

Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems [3]; Robust Estimation of Variance Components [4], Estimation and Use of Variance Components [5], Robust Variance Estimation in Linear Regression [6], Robust Variance Estimation for Random Effects Meta-analysis [7], Robust Variance Estimation in Meta-regression with Dependent Effect Size Estimates [8], Robust Estimation of Linear Mixed Models [9].

Robust estimates of variances and of their components have been continually applied in various fields [10] - [21] (a more detailed presentation can be seen in [22]).

Regarding a GPS measurement error as a random process and modeling the process realization structurally as a 2-way hierarchical classification with random effects, Perović [23] studied a robust solution for variance components in GPS measurements—PERG2FH method. Perović [22] also robustly estimated the parameters for both distributions of normally distributed observations that form Tukey’s mixed distribution.

From the said above it is seen that robust methods for estimation of Variance Components are studied. However, studies of Variance Components estimates on the basis of empirical variances distributions, except the standard procedure, have not been undertaken. For this reason the present author has devoted himself to VC research on the basis of these distributions, so that the here presented method was invented.

In the present paper a mixed distribution composed of two distributions of empirical variances is considered, where one of them is basic and the other one is contaminating, with an essential assumption that the variances of both distributions are obtained on the basis of primary normal distributions, as well as that they have the same degrees of freedom. The present author names this distribution PERG mixed-distribution of empirical variances (PERG—from initial letters of the author’s names: Perović Gligorije), which is presented in Figure 1 and described by means of the following model

F ( s 2 ) = ( 1 − ε ) F 1 ( s 2 ) + ε F 2 ( s 2 ) ,     ( 0 < ε < 0.5 ) (1)

¹PEROBVC is an abbreviation of the initial letters: Perović’s Robust Variance Components Estimation Model.

where F₁ is the basic distribution, F₂ the contaminating distribution, both of empirical variances s² with f d.f. and ε (0 < ε < 0.5) the contaminating degree.

On the basis of such a composition of the mixed distribution of empirical variances the author obtained a robust solution named Perović’s Robust Variance Components Estimation Model—PEROBVC¹ method for variances and for observation numbers for both distributions. In this way the method eliminates the influences of gross errors and outliers on the estimates of distribution parameters.

The key difference between the PEROBVC method and the existing ones is that distribution censoring is not used, but instead on the basis of structural decomposition in two distributions, except obtaining parameter estimates of the two distributions, the question of outliers is simultaneously solved.

Consequently, the PEROBVC method has the following properties:

1) Unbiased (exact) variance estimates for the basic distribution, as the most important property,

2) Unbiased (exact) variance estimates for the contaminating distribution,

3) Estimates of observation numbers for both distributions and in this way percentage estimates concerning fractions of basic and contaminating distributions in the mixed one.

The correctness of the PEROBVC method has been verified on exact (expected) values of some quantities from the mixed PERG distribution, as well as on examples of measurements in 2D control geodetic networks.

The variance estimate for the basic distribution obtained by applying the PEROBVC method is compared to its ML estimate.

The structure of the further presentation is the following: 1) establishing the PEROBVC method, 2) way of solving the formulated problem, 3) presentation of robust ML estimation method, 4) correctness verification for PEROBVC method by comparing its solution to the ML one and 5) verification on examples of geodetic measurements in to 2D control networks.

The idea of PEROBVC method has been presented in [24] (Section 32.14).

Consider a PERG mixed distribution (Figure 1) of empirical variances s ′ i 2 , i = 1 , 2 , ⋯ , n ′ and s ″ j 2 , j = 1 , 2 , ⋯ , n ″

F ( s 2 ) = ( 1 − ε ) F ( s ′ 2 ) + ε F ( s ″ 2 ) ,     ( 0 < ε < 0.5 ) (2)

with n ′ + n ″ = n and with the same degrees of freedom, f ′ i = f ″ j ≡ f . The variances are obtained from primary normally distributed observations with expectations

E [ s ′ i 2 ] = σ 1 2 ,   i = 1 , 2 , ⋯ , n ′ ,         E [ s ″ j 2 ] = σ 2 2 ,   j = 1 , 2 , ⋯ , n ″ ; σ 2 2 > σ 1 2 ,

where F ( s ′ 2 ) and F ( s ″ 2 ) are the distribution functions of the empirical variances s ′ 2 and s ″ 2 . The task is to estimate the variances σ 1 2 and σ 2 2 of both distributions on the basis of the mixed distribution with f known. In addition to the solving of this task, the method also yields estimates of the numbers of observations n ′ and n ″ for both component distributions F ( s ′ 2 ) and F ( s ″ 2 ) , i.e. the contaminating degree is estimated.

The point B divides the domain of s 2 into two sub-domains X and Y (Figure 1) and let be:

n ′ and n ″ —numbers of all observations, ( s ′ 2 and s ″ 2 ), within X and Y;

n ′ X and n ″ X —numbers of observations for s ′ 2 and s ″ 2 within X;

n ′ Y and n ″ Y —numbers of observations for s ′ 2 and s ″ 2 within Y,

n X + n Y = n       n ′ X + n ″ X = n X n ′ + n ″ = n         n ′ Y + n ″ Y = n Y } , (3)

The point B will be chosen in such a way that within the interval (0, B) the observations s ′ 2 dominate, whereas within [B, ¥) the observations s ″ 2 dominate from where it follow that it must be B = B_opt (also see Figure 1).

Here the point B is defined according as

B = s ( n X ) 2 + d / 2 , (4)

where d is the width of the rounding interval for observations s².

For the sake of simplifying the following designations are introduced:

( a )           w = s 2       with     f     d .f .             ⇒       w ′ = s ′ 2 ,         w ″ = s ″ 2 ( b )           y = f s 2 σ 2 = χ f 2                             ⇒       y ′ = f s ′ 2 σ 1 2 ,       y ″ = f s ″ 2 σ 2 2 } (5)

Then the density of the random variable _{s 2} -distribution can be written as

f ( s 2 ) = f ( w ) = k f − 1 σ − f f f / 2 w f / 2 − 1 e − w f / 2 σ 2 = f σ 2 f ( y ) , k f = 2 f / 2 Γ ( f 2 ) (6)

where Γ ( f / 2 ) is the gamma function of f/2. Then

F ( s 2 ) = F ( w ) = ∫ 0 w f ( w ) d w = ∫ 0 y f ( y ) d y = F ( y ) (7)

w f ( w ) d w = σ 2 f y f ( y ) d y       ⇒       ∫ 0 w w f ( w ) d w = σ 2 f ∫ 0 y y f ( y ) d y . (8)

From the equality condition for the ordinates of the two distribution densities, i.e. from the condition, one obtains optimal point B (B_opt)

B opt = 2 f ln ( n ′ n ″ ( σ 2 σ 1 ) f ) 1 σ 1 2 − 1 σ 2 2 . (9)

The estimates of the variance components for both distributions are obtained from the event-probability maximum

[ D ′ ( 1 ) ∧ ⋯ ∧ D ′ ( n ′ X ) ] ∧ ( s ″ 2 < B ) ∧ ( s ′ 2 > B ) ∧ [ D ″ ( n ″ X + 1 ) ∧ ⋯ ∧ D ″ ( n ″ ) ]

where D ′ ( 1 ) = s ′ ( 1 ) 2 < s ′ 2 < s ′ ( 1 ) 2 + d s ′ 2 = w ′ ( 1 ) < w ′ < w ′ ( 1 ) + d w ′ , ⋯ , D ″ ( n ″ ) = s ″ ( n ″ ) 2 < s ″ 2 < s ″ ( n ″ ) 2 + d s ″ 2 = w ″ ( n ″ ) < w ″ < w ″ ( n ″ ) + d w ″ , whose likelihood function, to the proportionality constant k, is

L = k ⋅ ∏ X f ( w ′ ( i ) ) ⋅ p ( w ″ < B ) ⋅ p ( w ′ > B ) ⋅ ∏ Y f ( w ″ ( i ) ) ,

where: ∏ X f ( w ′ ( i ) ) = ∏ 1 n ′ X f ( w ′ ( i ) ) ,.

In view of (5)-(8) one finds:

f ′ B = k f − 1 y ′ B f / 2 − 1 e − y ′ B / 2 f ″ B = k f − 1 y ″ B f / 2 − 1 e − y ″ B / 2 y ′ B = f ⋅ B σ 1 2 y ″ B = f ⋅ B σ 2 2 F ′ B = ∫ 0 y ′ B f ( y ′ ) d y ′ F ″ B = ∫ 0 y ″ B f ( y ″ ) d y ″ A ′ B = ∫ 0 y ′ B 1 f y ′ f ( y ′ ) d y ′ A ″ B = ∫ 0 y ″ B 1 f y ″ f ( y ″ ) d y ″ } . (10)

Furthermore lnL will be used there is:

ln L = c + ∑ X ln   f ( w ′ ( i ) ) + ln ( F ″ B ) n ″ X + ln ( 1 − F ′ B ) n ′ Y + ∑ Y ln f ( w ″ ( i ) ) , (11)

where c is a constant.

According to the asymptotic theory it is:

∑ X w ″ i → n ″ → ∞ p n ″ X ∫ 0 B w ″ f ( w ″ ) d w ″ , (12a)

∑ Y w ′ i → n ′ → ∞ p n ′ Y ∫ 0 B w ′ f ( w ′ ) d w ′ = n ′ Y σ 1 2 ( 1 − A ′ B ) . (12b)

Bearing in mind (10)-(12) and introducing the substitutions

( a )     ∑ X s ′ i 2 = ∑ X s i 2 − ∑ X s ″ i 2 , ∑ Y s ″ i 2 = ∑ Y s i 2 − ∑ Y s ′ i 2 ( b )     ∑ X s ″ i 2 = n ″ ∫ X s ″ 2 f ( s ″ 2 ) d s ″ 2 = n ″ σ 2 2 A ″ B ( c )     ∑ Y s ′ i 2 = n ′ ∫ Y s ′ 2 f ( s ′ 2 ) d s ′ 2 = n ′ σ 1 2 ( 1 − A ′ B ) }

from the necessary maximum condition: (a) ∂ ln L / ∂ σ 1 2 = 0 and (b) ∂ ln L / ∂ σ 2 2 = 0 we get the equations

( a ) − n ′ X 2 σ 1 2 + 1 2 σ 1 4 ∑ X s i 2 − n ″ X σ 2 2 2 σ 1 4 A ″ B + B n ′ Y f ′ B σ 1 4 ( 1 − F ′ B ) = 0 ( b ) − n ″ Y 2 σ 2 2 + 1 2 σ 2 4 ∑ Y s i 2 − B n ″ X f ″ B σ 2 4 F ″ B = 0 }

Their solution is given as simple iterations

σ 1 , k + 1 2 = 1 n ′ X , k [ ∑ X s i 2 − n ″ k σ 2 , k 2   A ″ B , k + 2 B n ′ Y , k f ′ B , k 1 − F ′ B , k ] , (13a)

σ 2 , k + 1 2 = 1 n ″ Y , k [ ∑ Y s i 2 − n ′ k σ 1 , k 2 ( 1 − A ′ B , k ) − 2 B n ″ X , k f ″ B , k F ″ B , k ] . (13b)

where the numbers n ′ X , n ′ Y , n ″ X and n ″ Y are determined from the expressions

( a )       n ′ X = n ′ F ′ B ,       n ′ Y = n ′ − n ′ X ( b )       n ″ X = n ″ F ″ B ,       n ″ Y = n ″ − n ″ X } , (14)

whereas n ′ and n ″ are determined as

F ′ B n ′ + F ″ B n ″ = n X ( 1 − F ′ B ) n ′ + ( 1 − F ″ B ) n ″ = n Y } (15)

Note 1. The initial values for the variance components can be assumed, for example, as:

σ 1 , 0 2 = 0.6 ⋅ ( 1 n ∑ 1 n s i 2 ) and σ 2 , 0 2 = 3 ⋅ ( 1 n ∑ 1 n s i 2 ) .

If the model assumptions are not satisfied, for example, if the distribution instead of a long tail has a shortened tail (from the right), then for one of the variances we obtain negative estimates. ∆

Note 2. Blunders in the observations (measurements with large gross errors) must be rejected before applying the PEROBVC method. For example, first find

s 2 = 1 n ∑ 1 n s i 2 and then s 0.9999 2 = 1 f χ f ; 0.9999 2 ⋅ s 2 .

Those s i 2 exceeding s 0.9999 2 are rejected, whereas this procedure is repeated once more with non-rejected results. ∆

The advantages of the PEROBVC method are:

1) Unbiased estimators for σ 1 2 and σ 2 2 , if B are close to B_opt, and

2) Minimal variances for σ 1 2 .

The disadvantages of the PEROBVC method are:

1) A sensitivity to the choice of the point B, (point B must be close to B_opt), which can result in negative estimates for either of the variances σ 1 2 or σ 2 2 , or for both, and

2) Sensitivity to the choice of the initial values for the variances, σ 1 2 and σ 2 2 , which, also, can result in negative estimates for one or both variances.

The stopping criterion for the iterative process: Let x k T = [ σ 1 , k 2 σ 2 , k 2 ] be the vector of parameter estimates in the k-th iteration and d the difference vector of these estimates from the (k + 1)-th and k-th iterations, d T = [ ( σ 1 , k + 1 2 − σ 1 , k 2 ) ( σ 2 , k + 1 2 − σ 2 , k 2 ) ] , then the iterations should be stopped if

‖ d ‖ < 10 − q ‖ x k ‖ ,     q ∈ { 5 , 6 , 7 , 8 , 9 } . (16)

In this case we shall use the method of Maximum Likelihood (ML).

The ML method is based on the assumption that in the domain X there exists only variances s ′ 2 ( s ′ 2 ≡ s 2 ; with E [ s 2 ] = σ 2 ) of the basic distribution, whereas in the Y interval in addition to basic variances exist the variances of contaminating distributions. Therefore the point B in Figure 1 will be the censoring point.

The ML solution is obtained from the probability maximum for the event

[ D ( 1 ) ∧ ⋯ ∧ D ( n X ) ] ∧ ( s 2 > B ) ,

where D ( i ) = s ( i ) 2 < s 2 < s ( i ) 2 + d s 2 , i = 1 , ⋯ , n X , one obtains the likelihood function (with proportionality constant k):

L = k ∏ i = 1 n X f ( s i 2 ) [ 1 − F ( B ) ] n Y .

Hence it follows

ln L ∝ ∑ X ln f ( s i 2 ) + n Y ln [ 1 − F ( B ) ] .

From the necessary condition ∂ ln L / ∂ σ 1 2 = 0 one obtains

− n X σ 2 + ∑ X s i 2 + 2 B n Y f ( y B ) 1 − F ( y B ) = 0 .

Their solution is given as simple iterations

σ k + 1 2 = 1 n X ∑ X s i 2 + 2 B n Y n X ⋅ f ( y B , k ) 1 − F ( y B , k ) , (17)

where

y B , k = f ⋅ B σ k 2 , F B , k = ∫ 0 y B , k f ( y ) d y , ( y = f s 2 σ 2 = χ f 2 , f ( y ) = k f − 1 y f / 2 − 1 e − y / 2 ) . (18)

The stopping criterion for the iterative process can be based on the parameter difference r σ = ( σ k + 1 2 − σ k 2 ) / σ k + 1 2 between two iterations:

| r σ | < 10 − c , c ∈ { 4 , 5 , 6 , 7 , 8 } , (19)

Example 1. By using the functions F-RAN1 and F-RAN2 [25], one simulates 10,000, (n = 10,000), samples (series) of size 4, f = 3, with F-RAN1 5000 samples are simulated ( n ′ = 5000 ), σ 1 2 = 1 and the variance estimate (from these 5000 samples) s 1 2 = 0.9997 , with F-RAN2 also 5000 samples ( n ″ = 5000 ), of the same size, with σ 2 2 = 2 and variance estimate (from these 5000 samples) s 2 2 = 1.9886 . Then a PERG mixed distribution (2), is formed ( ε = 50 % ).

Since the variances σ 1 2 and σ 2 2 are known then, according to (6), one can also find, B optimal, obtaining B_opt = 2.772589.

In Table 1, for 4 partitions (X, Y), we present the PEROBVC estimates of the variances σ 1 2 and σ 2 2 , as well as the contamination percentage ε , ( ε = n ″ / n ).

From Table 1 it is easily noticed that the variance components, as well as the numbers of observations for both distributions, are well estimated, i.e. their values are very close to the real ones. It is also noticed that the values of the ML estimates for σ 2 are smaller than those of the PEROBVC ones for σ 1 2 (where it is σ 2 ≡ σ 1 2 ). ▲

Example 2. In the 2D geodetic control network of object “TUZLA” [26] from the measurements of 328 (n = 328), horizontal angles, with a theodolite Wild T3, in 6 quick locks, in stable and unstable conditions, a mixed distribution of empirical variances (in ["²]), with f = 5 d.f., is obtained (Table 2).

In Table 3, for 7 partitions (X, Y), we present the estimates of the variances

and, as well as the contamination percentage ().

According to an a priori analysis for the horizontal-angle measurements in stable conditions, following Činklović [27], one obtains ["²], resulting in a very good agreement with 1.8109 ["²] obtained by using PEROBVC method based on the measurements.

Besides, on the basis of the results presented in Table 3 one concludes about a very high stability of the estimator for the variance component.

In Example 2 the average value from the ML estimates for, (), is also smaller than the average value of the PEROBVC estimates for (1.6753 < 1.8109).

It should be noted that for the partition the numbers of observations and are well estimated, although B = 1.8 is significantly smaller than B_opt = 8.18. ▲

Note 3. In Examples 1-2 the ML estimates for are smaller than the corresponding PEROBVC estimates for, (), because higher values of empirical variances are in the end of the distribution so that they do not enter the ML estimate for. ∆

On the basis of applications of the PEROBVC method in the treatment of real geodetic measurements, Example 2, the present author concludes that the variance estimator of the basic distribution, has a good stability.

Generally, the PEROBVC method offers good estimates, especially for, which is illustrated by Examples 1-2.

On the basis of the obtained results in Examples 1-2 we can conclude the following:

1) On the basis of exact (expected) values from Example 1 the validity of the PEROBVC method in the variance estimation, as well as the estimates of numbers of observations, for both distributions in the PERG mixed distribution of empirical variances is confirmed. Here the variance estimates for both distributions, basic and contaminating ones, are correct; i.e. their values are exact.

2) On the basis of realistic measurements for the horizontal angles in the geodetic 2D control network from Examples 2 good (satisfactory) parameter estimates for both distributions are also confirmed.

3) In Example 2 the variance estimate for the basic distribution is confirmed through the result of the a priori analysis.

Dr ZORICA Cvetković from the Astronomical Observatory in Belgrade is the author of the FORTRAN programmes who performed the calculations in the examples. The OBJEKTIV GEO D.O.O. Company from Belgrade (Serbia) has financially supported the publishing of the article.

The author declares no conflicts of interest regarding the publication of this paper.

Perović, G. (2020) Robust Variance Components Estimation in the PERG Mixed Distributions of Empirical Variances—PEROBVC Method. Open Journal of Statistics, 10, 640-650. https://doi.org/10.4236/ojs.2020.104038