First consider the model (1) and assume that. We have then the system of linear equations

and for the noise-free observations.

Suppose that the system (6) is compatible, i.e. there exists such that. In what follows let

, , i.e. is the component of, is the row-vector of.

The problem is to obtain a simple recursive procedure to compute a solution of the system (6) when the dimension of is very high.

To find a solution to Equation (6), let us introduce the following system of recursive equations

Mention that the system is compatible if where is a linear space spanned by the columns of. Throughout of the paper, for definiteness, let,.

Theorem 1. Suppose the system (6) is compatible. Then for any finite and symmetric positive definitive (SPD) matrix we have.

In order to prove Theorem 1 we need

Lemma 1. The following equalities hold

Proof. By induction. We have for,

since.

Let the statement be true for some. We show now that it is true also for. As the statement is true for, it implies,. We have to prove

Substituting into, taking into account the form of one sees that as, it implies, (End of Proof).

Lemma 2. The following equalities hold

Proof. By induction. We have for,. As, it is evident that.

Let the statement be true for some. We show now that it is true also for. As the statement is true for, it implies,. We have to prove,. From the definition of,

However from Lemma 1, as, for all. (End of Proof).

Proof of Theorem 1.

Theorem follows from Lemma 2 since under the conditions of Theorem, from Equation (9) for it follows, or. (End of Proof).

Corollary 1. Suppose the rows of are linearly independent. Then under conditions of Theorem 1,.

Proof. By induction. The fact that for, implies at least the null subspace of has one nonzero element hence. We show now that it is impossible that.

Suppose that, For simplicity, let. It means that there exist linearly independent vectors such that any element from the subspace can be represented on the basis of these elements. As to the matrix

its subspace has the dimension 1 hence any element from can be represented on the basis of some vector. Thus any element from the subspace with can be represented as a linear combination of elements. On the other hand, as is non-singular, any element of must be represented as a linear combination of linearly independent vectors. It contradicts the fact that any element from could be written as a linear combination of linearly

independent elements. We conclude that it is impossible hence. The same argument is true for.

Suppose now Corollary is true for and we have to prove that it holds for. For

we have,. From Lemma 1 it follows, hence the null subspace has the dimension (as all the rows of are linearly independent). It follows that the dimension of the subspace is at least less or equal to.

By the same way as proved for one can show that it is impossible hence (End of Proof).

Comment 1. By verifying the rank of, Corollary 1 allows us to check if the computer code is correct. In particular if is non-singular, at the end of the iterative procedure the matrix should be zero. The recursive Equations (7a)-(7c) then yield the unique solution of the equation after iterations.

Using the result (8) in Lemma 1, it is easy to see that:

Corollary 2. Suppose is linearly dependent on. Then in (7),.

Corollary 3. Suppose are linearly independent, is linearly dependent on. Then under the conditions of Theorem 1,

Corollary 3 follows from the fact that when is linearly dependent on from Corollary 2, and hence by Corollary 1,.

Comment 2. Equation (7c) for is similar to that given in (3.14.1) in [3] since from Corollary 2 it follows automatically if is linearly dependent on. Their difference is that in (7c) the initial may be any SPD matrix whereas in (3.14.1) of [3] it is assumed.

In the next section we shall show that the fact may be any SPD matrix is important to obtain the optimal in mean squared estimator for.

Theorem 1 says only that Equations (7a)-(7c) give a solution to (1). We are going now to study the question on whether a solution of Equations (7a)-(7c) is optimal, and if yes, in what sense? Return to Equations (1)-(3) and assume that,. The optimal estimator (4) is then given in the form

where is the pseudo-inversion of. Consider first the equation. In this case, (10) yields the optimal estimator (obtained on the basis of the first observation)

and one can prove also that the mean square error for is equal to

If we apply Equation (3.14.1) in [3] , then instead of we have

For simplicity, let. Comparing Equation (12) with Equation (11) shows that if is the orthogonal projection of onto the subspace spanned by, the estimate belongs to the subspace spanned by. Thus the algorithm (11) takes into account the fact that we known a priori belongs to the space. This fact is very important when the number of observations is much less than the number of the estimated parameters as it happens in oceanic data assimilation: today usually,.

In [4] a similar question has been studied which concerns the choice of adequate structure for the Error Covariance Matrix (ECM).

We prove now a more strong result saying that all the estimates, are projected onto the space.

Theorem 2. Consider the algorithm (7). Suppose. Then all the estimates, belong to the space spanned by the columns of, i.e..

Proof. For the statement is evident as shown above.

Suppose the statement is true for some. We will show that it is true also for.

Really as, it is sufficient to show that From, , as it follows that. But hence the columns of must belong to. Again from the equation for in Equation (7) we have. It proves since (End of proof).

Theorem 2 says that by specifying the algorithm (7) will produce the estimates belonging to the subspace. Specification of the priori matrix plays the most important task if we want the algorithm (7) to produce the estimate with high quality.

Comment 3

1) If does not belong to, by considering in place of Theorem 3 remains valid for the algorithm (7) written for. In this case at the first iteration, as represents the priori error for, it is natural for the algorithm to seek the correction (i.e., the estimate for the error) belonging to the space. In what follows, unless otherwise stated, for simplicity we assume.

2) Theorem 2 says that there is a possibility to regularize the estimate when the number of observations is less than the number of estimated parameters by choosing. Thus the algorithm can be considered as that which finds the solution under the constraint. In the procedure in Albert (1972) putting means that there is no constraint on hence the best way to do is to project orthogonally onto subspace of.

Suppose is a random variable having the mean and covariance matrix. We have then the following result.

Theorem 3. Suppose is a random variable having the mean and the covariance matrix. Then generated by the recursive Equation (7) is an unbiased and minimum variance estimate for in the class of all unbiased estimates linearly dependent on and.

Proof. Introduce for the system (6),

and the class of all estimates linearly dependent on and.

The condition for unbiasedness of is

or

from which follows. Substituting this relation into leads to

It means that all the estimate in Equation (14) is unbiased.

Consider the minimization problem

We have

Taking the derivative of with respect to implies the following equation for finding,

from which follows one of the solutions

If now instead of Equation (13) we consider the system

and repeat the same proof, one can show that the unbiased minimum variance estimate for is given by

Using the properties of the pseudo-inverse (Theorem 3.8 [3] ), one can prove that

Thus for a very small the estimate is unbiased minimum variance which can be made as close as possible to.

On the other hand, applying Lemma 1 in [5] for the case of uncorrelated sequence, one can show that

Letting one comes to in (7) (End of proof).

The algorithm (18a)-(18d) thus yields an unbiased minimal variance (UMV) estimates for in the situation when represents the observation noise variance. We want to stress that these algorithms produce the UMV estimates only if where is the true covariance of the error before arriving,.

In the field of data assimilation in meteorology and oceanography usually the state vector is of very high dimension, the orders of [2] . This happens because is a collection of several variables defined in the three dimensional grid. If the algorithm (7) allows to overcome the difficulties with the high dimension of the observation vector (each iteration involves one component of), due to high dimension of , it is impossible to handle Equation (7c) to evaluate the matrices (with the number of elements). This section is devoted to the question on how one can overcome such difficulties.

Let us consider the eigen-decomposition for [6]

In (19) the columns of are the eigenvectors of and is diagonal with the elements at the diagonal?the eigen-values of. Let,. If we put in the algorithms (7) or (18), then the algorithm (7), for example, will yield the best estimate for projected in the subspace. Let has the dimension,. Let

.

In this section we will show importance of the regularization factor in the form of the priori covariance in the design of a filter for systems of very high dimension.

The numerical model used in this experiment is the MICOM (Miami Isopycnic Coordinate Ocean Model) which is exactly as that presented in [8] . We recall only that the model configuration is a domain situated in the North Atlantic from to and to. The grid spacing is about in longitude and in latitude, requiring the horizontal mesh;. The distance between two points,. The number of layers in the model. We note that the state of the model where is the thickness of the layer, , are two velocity components. The “true” ocean is simulated by running the model from “climatology” during two years. Each ten days the sea-surface height (SSH) are stored at the grid points; which are considered as observations in the assimilation experiment. The sequence of true states will be available and allows us to compute the estimation errors. Thus the observation operator is constant at all assimilation instants.

The assimilation experiment consists of using the SSH to correct the model solution, which is initialized by some arbitrarily chosen state resulting from the control run.

The different filters are implemented to estimate the oceanic circulation. It is well known that determining the filter gain is one of the most important tasks in the design of a filter. As for the considered problem it is impossible to apply the standard Kalman filter [9] since in the present experiment, and the number of elements in the ECM is of order. At each assimilation instant, the estimate for the system state in all filters is computed in the form (10) with the corresponding ECM. As the number of observations is largely inferior to the dimension of the system state, the choice of as a regularization factor has a great impact on the quality of the produced estimates. In this assimilation experiment the following filters will be employed. First the Prediction Error Filter (PEF) whose ECM is obtained on the basis of leading real Schur vectors [8] . Parallelly two other filters, one is the Cooper-Haines filter (CHF) [10] and another is an EnOI (Ensemble based Optimal Interpolation) filter [11] will be used. Mention that the ECM in the CHF is obtained on the basis of the principle of a vertical rearrangement of water parcels (see also [8] ). The method conserves the water masses and maintains geostrophy. The main difference between PEF and EnOI is lying in the way to generate the ensembles of Prediction Error (PE) samples. In the PEF, the ensemble of PE samples is generated using the sampling procedure described in [8] (and it will be denoted as En(PEF)). As for the EnOI, the ensemble of background errors samples (the term used in [8] ) and will be denoted by En(EnOI)) will be used. The elements of En(EnOI) are constructed according to the method in [11] . It consists of using 2-year mean of true states as the background field and the error samples are calculated as differences between individual 10-day true states during this period and the background.

According to Corollary 4.1 in [4] , using the hypothesis on separable vertical-horizontal structure for the ECM, we represent where, are the ECM of vertical and horizontal variables respectively. In the case of sea-surface height observations, from the representation, the gain filter can be represented in the form

where denotes the Kronecker product. The gain allows the correction available at the surface to propagate into all vertical subsurface layers. As to, it represents an operator of (horizontal) optimal interpolation which interpolates the observations over all horizontal grid points at the surface. Mention that the elements of and the correlation length parameter in are estimated by minimizing the mean distance between the data matrix and using a simultaneous perturbation stochastic approximation algorithm [12] . The data matrix is obtained from samples of the leading real Schur vectors as described in [8] .

Figure 2 shows the estimated vertical coefficients, obtained on the basis of the ECM spanned by the elements of En(PEF). It is seen that the estimates converge quite quickly. The estimated vertical gain coefficients computed on the basis of the ECM from two ensembles En(PEF), En(EnOI) at the iteration are

We remark that all the gain coefficients in two filters are of identical sign but the elements of are of much less magnitudes than that of. It means that the EnOI will make less correction (compared to the PEF) to the forecast estimate. Two gains in (29a), (29b) will be used in the two filters PEF and EnOI to assimilate the observations.

The vertical gain coefficients for the CHF are taken from [8] and are equal to

In Figure 3 we show the instantaneous variances of the SSH innovation produced by three filters EnOI, CHF and PEF. It is seen that initialized by the same initial state, if the innovation variances in EnOI, CHF have a tendency to increase, this error remains stable for the PEF during all assimilation period. At the end of assimilation, the PE in the CHF is more than two times greater than that produced by the PEF. The EnOI has produced poor estimates, with error about two times greater than the CHF has done.

For the velocity estimates, the same tendency is observed as seen from Figure 4 for the surface velocity PE

errors. These results prove that the choice of ECM as a regularization factor on the basis of members of the En(PEF) allows to much better approach the true system state compared to that based on the samples taken from En(EnOI) or to that constructed on the basis of the physical consideration as in the CHF. The reason is that the members of En(PEF) by construction [8] are samples of the directions along which the prediction error increases most rapidly. In other words, the correction in the PEF is designed to capture the principal important components in the decomposion of the covariance of the prediction error.