Consider the overdetermined linear system A x ≈ b , where A ∈ ℝ m × n and b ∈ ℝ m , and the total least squares (TLS) problem can be formulated as (see [1] )

m i n E , f ‖ [ E f ] ‖ F ,     subject     to     ( A + E ) x = b + f . (1)

However, in many linear parameter estimation problems, the error of data matrix A on the left side of the approximate system may be scaled. In order to maximize the accuracy of the estimated parameters x, the case where the scaling factor is used to weight some columns of error matrix in data matrix A is naturally considered when estimating parameters x using the TLS approach. From the point of view, Liu, Wei and Chen [2] proposed the concept of the matrix-scaled total least squares (MSTLS) problem with single right-hand side in 2019. Inspired by [3], Liu, Wei and Chen [2] transformed the MSTLS problem with single right-hand side into the weighted TLS (WTLS) problem.

As a continuation of their work, we extend the MSTLS problem with single right-hand side to multiple right-hand sides as follows.

m i n E , F ‖ γ E 1 E 2 F ‖ F ,     subject   to     ( A 1 + γ E 1 ) X 1 + ( A 2 + E 2 ) X 2 = B + F , (2)

where X = [ X 1 X 2 ] ∈ ℝ n × d , A 1 ∈ ℝ m × n 1 , A 2 ∈ ℝ m × n 2 , n 1 + n 2 = n and B ∈ ℝ m × d ( m ≥ n + d ) .

When d = 1 , our model (2) degenerates to the single right-hand case.

The condition number is a measure of the sensitivity of the solution to input data perturbation. Therefore, condition numbers play an important role in numerical analysis. Many scholars have studied the TLS problem with multiple right-hand sides: [4] [5] [6] [7] [8] studied the sufficient conditions or/and necessary conditions for its solvability. When the TLS problem is solvable, Huang, Yan and Yu discussed the solution set and minimal norm solution of the TLS problem, see [4] [9]. Recently, based on singular value decomposition (SVD) of augmented matrix, many scholars have discussed the solution set and minimal norm solution of the TLS problem. In [10], Hnětynková et al. proposed the TLS problem with multiple right-hand sides, and gave the conditions for the existence and expression of the solutions. In [11], Hnětynková et al. extended the concept of the core problem of the TLS problem with single right-hand side proposed by Paige and Strakoš in [12] to the TLS problem with multiple right-hand sides. The perturbation analysis of the solution of the TLS problem with multiple right-hand sides can be found in references [7] [8] [13] [14] [15]. Zheng, Meng and Wei gave the exact expressions and their upper bounds of normwise, mixed and componentwise condition numbers of TLS problem with multiple right-hand sides in [16] [17]. These results extend the condition number theory of TLS problem with single right-hand side. In addition, by making use of the perturbation results in the above references, Liu et al. [18] [19] derived closed formulae for condition numbers of the total least squares problem with linear equality constraint problem.

As far as we know, the perturbation analysis of the MSTLS problem has not been systematically performed in literature. In this paper, we consider the perturbation analysis of the MSTLS problem (2). The relative normwise condition number, the mixed condition number and the componentwise condition number are derived. Our analysis can be seen as a unified treatment of the mentioned approaches in [3] and [16].

Throughout this paper, for given positive integers m and n, we denote by ℝ m × n the space of all m × n real matrices, and I n stands for the identity matrix of order n. ‖ ⋅ ‖ 2 , ‖ ⋅ ‖ ∞ and ‖ ⋅ ‖ F denote the 2-norm, ∞-norm, and Frobenius norm of their arguments, respectively. Given a matrix X = ( x i j ) ∈ ℝ m × n , ‖ X ‖ m a x , X T , X † and σ i ( X ) denote the “max” norm given by ‖ X ‖ m a x = m a x i , j | x i j | , the transpose, the Moore-Penrose inverse and the i-th largest singular value of X, respectively. | X | is the absolute value of the matrix X, whose entries are | x i j | . Moreover, for Y = ( y i j ) , Z = ( z i j ) ∈ ℝ m × n , Y Z is an entry-wise division; that is, Y Z = ( y i j / z i j ) , or Y./Z in Matlab notation. Here, ξ/0 is interpreted as zero if ξ = 0 and ∞ otherwise. For matrices A = [ a 1 ⋯ a n ] = ( a i j ) ∈ ℝ m × n , vec ( A ) = [ a 1 T ⋯ a n T ] T ∈ ℝ m n × 1 and A ⊗ B = [ a i j B ] denotes the Kronecker product of A and any matrix B. Let Π ( m , n ) = ∑ i = 1 m   ∑ j = 1 n   E i j ⊗ E i j T , where E i j ∈ ℝ m × n has an entry 1 in position ( i , j ) and all other entries are zeros.

The organization of this paper is as follows. In Section 2, we present some necessary preliminaries, including the explicit expression for minimum W γ − 2 -norm MSTLS solution under some mild conditions and some important lemmas. In Section 3, we derive the normwise, mixed and componentwise condition numbers and their computable upper bounds of the MSTLS solution. All these results can reduce to those of the TLS problem which were given in [16]. Two numerical examples are tested in Section 4 to demonstrate the tightness of the derived upper bounds. Some concluding remarks are given in Section 5. In the appendix, we present a power method for calculating the normwise condition number of the MSTLS solution X MSTLS similar to the one in [20].

In this section, we give an explicit expression for the minimum W γ − 2 -norm solution of the MSTLS problem with multiple right-hand sides (2) under some mild conditions.

Along the similar lines as in [2] [3], we can see that (2) is equivalent to the following WTLS model:

m i n E , F ‖ [ E 1 E 2 F ] ‖ F ,     subject   to   [ ( γ A 1 , A 2 ) + ( E 1 , E 2 ) ] W γ − 1 X = B + F , (3)

where W γ = [ γ I n 1 0 0 I n 2 ] . Let

A γ = [ A B ] W ¯ γ     with     W ¯ γ = [ W γ 0 0 I d ] ,

and the SVD of A γ be

A γ = U Σ V T = [ U 1 U 2 ] [ Σ 1 0 0 Σ 2 ] [ V 1 T V 2 T ] , (4)

where U 1 ∈ ℝ m × p , U 2 ∈ ℝ m × ( m − p ) , V 1 = n d [ V 11 V 21 ] ∈ ℝ ( n + d ) × p , V 2 = n d [ V 12 V 22 ] ∈ ℝ ( n + d ) × ( n + d − p ) , Σ 1 = diag ( σ 1 , σ 2 , ⋯ , σ p ) ∈ ℝ p × p and Σ 2 = diag ( σ p + 1 , σ p + 2 , ⋯ , σ n + d ) ∈ ℝ ( m − p ) × ( n + d − p ) .

The following theorem gives the minimum W γ − 2 -norm solution of (2).

Theorem 2.1 Let the SVD of A γ be given as in (4). If

p ≤ n ,     σ p > σ p + 1     and     rank ( V 22 ) = d , (5)

thenthe MSTLS problem with multiple right-hand sides (2) has the minimum W γ − 2 -norm solution:

X MSTLS = − W γ V 12 V 22 † , (6)

wherethe weighted W γ − 2 -norm is defined by ‖ X ‖ W γ − 2 = trace ( X T W γ − 2 X ) .

Proof. By ( [8], (2.3)), we can get the minimum F-norm solution to (3) under the condition (5):

W γ − 1 X MSTLS = − V 12 V 22 † ,

which leads to the minimum W γ − 2 -norm solution of (2)

X MSTLS = − W γ V 12 V 22 † .

Therefore, we complete the proof.

In this paper, all discussions are based on the solvability conditions (5). Next we give some lemmas which will be used in the next analysis.

Lemma 2.1 ( [21], Chapt. 2) Let B ∈ ℝ m × n , C ∈ ℝ p × q , X ∈ ℝ n × p . Then

( B ⊗ C ) T = B T ⊗ C T , ‖ B ⊗ C ‖ 2 = ‖ B ‖ 2 ‖ C ‖ 2 ,

vec ( B X C ) = ( C T ⊗ B ) vec ( X ) ,

vec ( B T ) = Π ( m , n ) vec ( B ) ,

Π ( p , m ) ( C ⊗ B ) = ( B ⊗ C ) Π ( n , q ) .

If an orthogonal matrix of size n is partitioned into a 2 × 2 block form, some interesting connections and properties among these four submatrix blocks are provided in the lemma below.

Lemma 2.2 ( [16], Lemma 2.1) Let Q = [ Q 11 Q 12 Q 21 Q 22 ] be an n-by-n orthogonal matrix with a 2-by-2 partitioning. Then

1) Q 11 has full column (row)rank if and only if Q 22 has full row (column) rank;

2) ‖ Q 11 † ‖ 2 = ‖ Q 22 † ‖ 2 , ( Q 11 T ) † = Q 11 − Q 12 Q 22 † Q 21 , ( Q 11 T ) † Q 21 T = − Q 12 Q 22 † .

Since the MSTLS solution is closely related to the matrix V of right singular vectors of augmented matrix, the first-order perturbation analysis of V will play an important role in next discussions. The next lemma gives the first-order perturbation analysis of V.

Lemma 2.3 ( [16], Lemma 2.2) Let A γ have the SVD in (4) with σ p > σ p + 1 and ‖ E ‖ 2 be sufficiently small. Define

R = ( Σ 1 2 ⊗ I n + d − p − I p ⊗ ( Σ 2 T Σ 2 ) ) − 1 [ I p ⊗ Σ 2 T Σ 1 ⊗ I n + d − p ] ,

S = [ V 1 T ⊗ U 2 T Π ( p , n + d − p ) ( V 2 T ⊗ U 1 T ) ] .

Then the matrix V ˜ = [ V ˜ 1 V ˜ 2 ] of right singular vectors of A ˜ γ = A γ + E satisfies

V ˜ 1 = ( V 1 + V 2 P ) ( I + P T P ) − 1 2 ,   V ˜ 2 = ( V 2 − V 1 P T ) ( I + P P T ) − 1 2 ,

where P ∈ ℝ ( n + d − p ) × p is given by

vec ( P ) = R S   vec ( E ) + O ( ‖ E ‖ 2 2 ) .

Moreover, if V 22 has full row (column)rank,then V ˜ 22 has full row (column) rank.

Let A ˜ = [ A ˜ 1 A ˜ 2 ] = A + Δ A and B ˜ = B + Δ B , where Δ A = [ Δ A 1 Δ A 2 ] and Δ B are the perturbations of the input data A and B, respectively.

Consider the perturbed MSTLS problem with multiple right-hand sides

m i n E , F ‖ ( γ E 1 E 2 F ) ‖ F ,     subject   to   ( A ˜ 1 + γ E 1 ) X 1 + ( A ˜ 2 + E 2 ) X 2 = B ˜ + F . (7)

Similarly, (7) is also treated as a perturbed WTLS problem. Let A ˜ γ = [ A ˜ B ˜ ] W ¯ γ , and the SVD of A ˜ γ be

A ˜ γ = U ˜ Σ ˜ V ˜ T = [ U ˜ 1 U ˜ 2 ] [ Σ ˜ 1 0 0 Σ ˜ 2 ] [ V ˜ 11 V ˜ 12 V ˜ 21 V ˜ 22 ] T , (8)

where U ˜ , Σ ˜ and V ˜ are partitioned as in U, Σ and V in (4), respectively.

When the norm ‖ [ Δ A Δ B ] ‖ F of the perturbations is sufficiently small, then perturbation analysis of singular values can ensure that the perturbed MSTLS problem with multiple right-hand sides (7) has the minimum W γ − 2 -norm solution:

X ˜ MSTLS = − W γ V ˜ 12 V ˜ 22 † . (9)

Let Δ X = X ˜ MSTLS − X MSTLS . Now we introduce definitions of the normwise, mixed and componentwise condition numbers for X MSTLS as follows.

Definition 3.1 The absolute normwise condition number for X MSTLS is defined by

κ MSTLS abs ( X MSTLS , A γ ) : = l i m ε → 0 l i m ‖ Δ A γ ‖ F ≤ ε ‖ A γ ‖ F ‖ Δ X ‖ F ‖ Δ A γ ‖ F , (10)

therelative normwise condition number for X MSTLS is defined by

κ MSTLS rel ( X MSTLS , A γ ) : = l i m ε → 0 l i m ‖ Δ A γ ‖ F ≤ ε ‖ A γ ‖ F ‖ Δ X ‖ F ε ‖ X MSTLS ‖ F , (11)

themixed condition number for X MSTLS is defined by

κ MSTLS mix ( X MSTLS , A γ ) : = l i m ε → 0 s u p | Δ A W γ | ≤ ε | A W γ | | Δ B | ≤ ε | B | ‖ Δ X ‖ m a x ε ‖ X MSTLS ‖ m a x , (12)

andthe componentwise condition number for X MSTLS is defined by

κ MSTLS com ( X MSTLS , A γ ) : = l i m ε → 0 s u p | Δ A W γ | ≤ ε | A W γ | | Δ B | ≤ ε | B | 1 ε ‖ Δ X X MSTLS ‖ m a x . (13)

Definition 3.1 only provides general descriptions of the various condition numbers. It is usually hard to give their exact computable formulae. However, if X MSTLS is a differentiable function with respect to the data, the condition numbers defined in Definition 3.1 can be exactly expressed in derivatives. We start from the differentiability of X MSTLS .

Let the SVD of the matrices A γ be given as in (4). To derive the exact formulae of the condition numbers of X MSTLS = − W γ V 12 V 22 † , we define the mapping ϕ : ℝ m ( n + d ) → ℝ n d by

ϕ ( c ) = vec ( X MSTLS ) ,

where c = vec ( A γ ) . It follows from Lemma 2.3 that ϕ is continuous in a neighborhood of c. Using the definitions of the condition numbers of the mapping ϕ at a fixed point c [22] [23] [24], and following [23] [24] [25], if ϕ is Fréchet differentiable in a neighborhood of c, we have

κ MSTLS abs ( X MSTLS , A γ ) = κ MSTLS abs ( ϕ , c ) = ‖ ϕ ′ ( c ) ‖ 2 , (14)

κ MSTLS rel ( X MSTLS , A γ ) = κ MSTLS rel ( ϕ , c ) = ‖ ϕ ′ ( c ) ‖ 2 ‖ c ‖ 2 ‖ ϕ ( c ) ‖ 2 , (15)

κ MSTLS mix ( X MSTLS , A γ ) = κ MSTLS mix ( ϕ , c ) = ‖ | ϕ ′ ( c ) | | c | ‖ ∞ ‖ ϕ ( c ) ‖ ∞ , (16)

κ MSTLS com ( X MSTLS , A γ ) = κ MSTLS com ( ϕ , c ) = ‖ | ϕ ′ ( c ) | | c | | ϕ ( c ) | ‖ ∞ , (17)

where ϕ ′ ( c ) denotes the Fréchet derivative of ϕ at point c.

Before deriving the explicit expression for condition numbers, we give a useful lemma which proves that ϕ is Fréchet differentiable in a neighborhood of c = vec ( A γ ) and gives the explicit expression for ϕ ′ ( c ) .

Lemma 3.1 Under conditions (5), the mapping ϕ defined above is continuous and Fréchet differentiable at c = vec ( A γ ) . Moreover, its Fréchet derivative has the expression

ϕ ′ ( c ) = ( H 1 + H 2 ) R S (18)

inwhich R and S are defined as in Lemma 2.3,and

H 1 = ( ( V 22 V 22 T ) − 1 V 21 ) ⊗ ( W γ V 12 F V 22 ) ,     H 2 = ( ( V 22 † ) T ⊗ W γ ( V 11 T ) † ) Π ( n + d − p , p )

with F V 22 = I − V 22 † V 22 .

Proof. We only prove the second statement since the first part is trivial. According to Lemma 2.3, it follows that there exists a proper matrix P such that

V ˜ 12 = ( V 12 − V 11 P T ) ( I + P P T ) − 1 2 ,   V ˜ 22 = ( V 22 − V 21 P T ) ( I + P P T ) − 1 2 .

Since V ˜ 22 has full row rank by Lemma 2.3, we have

X ˜ MSTLS = − W γ V ˜ 12 V ˜ 22 † = − W γ V ˜ 12 V ˜ 22 T ( V ˜ 22 V ˜ 22 T ) − 1 = − W γ ( V 12 − V 11 P T ) ( I + P P T ) − 1 ( V 22 − V 21 P T ) T     × ( ( V 22 − V 21 P T ) ( I + P P T ) − 1 ( V 22 − V 21 P T ) T ) − 1 .

Using Lemma 2.2 and only retaining the first-order terms give

X ˜ MSTLS = X MSTLS + W γ V 12 ( I − V 22 † V 22 ) P V 21 T ( V 22 V 22 T ) − 1     + W γ ( V 11 − V 12 V 22 † V 21 ) P T V 22 † + O ( ‖ Δ A γ ‖ 2 2 ) = X MSTLS + W γ V 12 F V 22 P V 21 T ( V 22 V 22 T ) − 1     + W γ ( V 11 T ) † P T V 22 † + O ( ‖ Δ A γ ‖ 2 2 ) ,

which together with Lemma 2.1 and Lemma 2.3 leads to

ϕ ( vec ( A ˜ γ ) ) − ϕ ( vec ( A γ ) ) = vec ( X ˜ MSTLS − X MSTLS ) = ( ( ( V 22 V 22 T ) − 1 V 21 ) ⊗ ( W γ V 12 F V 22 )         + ( ( V 22 † ) T ⊗ W γ ( V 11 T ) † ) Π ( n + d − p , p ) ) vec ( P ) + O ( ‖ Δ A γ ‖ 2 2 ) = ( H 1 + H 2 ) R S   vec ( Δ A γ ) + O ( ‖ Δ A γ ‖ 2 2 ) .

Consequently, the Fréchet derivative of ϕ at c = [ a T b T ] T is given by

ϕ ′ ( c ) = ( H 1 + H 2 ) R S ,

which gives the desired result.

In this section, we present three numerical experiments to illustrate that the tightness of the upper bound estimates on the absolute normwise, mixed and componentwise condition numbers of the MSTLS solution and the operability of Algorithm 1, respectively. All of the following numerical experiments are performed via MATLAB R2014b in a laptop with AMD A10-7300 Radeon R6, 10 Compute Cores 4C + 6G by using double precision. Each figure in the following tables is the average of 500 experiments.

Example 4.1 Let

W ¯ γ = [ γ 0 0 I 3 ] ,

and A γ be given by its SVD decomposition

A γ = [ A B ] W ¯ γ = U   diag ( 3 , 2 , 1 , 1 ) ( 1 2 [ 1 1 − 1 1 1 1 1 − 1 1 − 1 1 1 1 − 1 − 1 − 1 ] T ) ,

where A , B ∈ ℝ 4 × 2 and U is an arbitrary 4-by-4 orthogonal matrix. This example is inspired by ( [16], Example 3.7).

We partition the unitary matrix V:

V = [ V 11 V 12 V 21 V 22 ] = ( 1 2 [ 1 1 − 1 1 1 1 1 − 1 1 − 1 1 1 1 − 1 − 1 − 1 ] ) .

We know σ 1 > σ 2 and rank ( V 22 ) = 2 = d . Hence the approximate linear system (2) has the MSTLS solutions and the normwise condition number of its minimum W γ − 2 -norm solution X MSTLS satisfies Table 1, which means that the upper bound κ ¯ MSTLS abs can be reached and therefore is optimal.

Example 4.2 Let

A 0 W γ = U   diag ( 10,7,7,3,2,1, σ 7 ( A 0 ) ,0.0005,0.0001,0.00005 ) V T ∈ ℝ 50 × 10

with U ∈ ℝ 50 × 50 and V ∈ ℝ 10 × 10 being arbitrary orthogonal matrices and W γ = [ γ I 4 0 0 I 6 ] . We choose B 0 ∈ ℝ 50 × 4 such that B 0 = A 0 W γ Y 1 + Y 2 and A γ = [ A 0 W γ B 0 ] + [ E F ] , where Y 1 , Y 2 , E, and F have entries from the standard normal distribution such that σ p > σ p + 1 and rank ( V 22 ) = 4 with p = 4 .

This example is a modification from [14] and Zheng et al. ( [16], Example 5.2) used the similar example to compare the mixed and componentwise condition numbers with their corresponding upper bounds of the TLS problem with multiple right-hand sides. Next, we will show the exact condition numbers of X MSTLS and its upper bounds when γ = 1 e + 15,1 e + 10,1 e + 5,1 e − 5,1 e − 10,1 e − 15 , respectively.

We compute the mixed condition number κ MSTLS mix , the componentwise condition number κ MSTLS com of X MSTLS and their corresponding upper bounds κ ¯ MSTLS mix and κ ¯ MSTLS com with σ 7 ( A 0 ) = 0.001 , 0.01, 0.1 and 1, and report the results in Table 2, Table 3.

As shown in Table 2 and Table 3, we can see that the upper bounds are at most two orders of magnitude larger than the corresponding exact condition number. That is, the upper bounds in Theorem 3.4 can estimate their corresponding condition numbers well.

Example 4.3 Let A γ be given as in Example 4.1. Take γ = 1 e + 5 , and σ 7 ( A 0 ) = 0.0005 + ( 1 e − 9 ) , 0.0005 + ( 1 e − 7 ) , 0.0005 + ( 1 e − 4 ) , respectively.

The quantity σ p − σ p + 1 measures the distance of our problem to nongenericity, and we have in exact arithmetic σ p − σ p + 1 . Then by varying σ p + 1 , we can generate different MSTLS problems, and by considering values of σ 7 ( A 0 ) , it is possible to study the behavior of the MSTLS condition number in the context of close-to-nongeneric problems. Firstly, we compare in Table 4 the exact condition number κ MSTLS abs given in Theorem 3.1, the upper bound κ ¯ MSTLS abs given in Theorem 3.2. We also report the condition number computed by Algorithm 1, denoted by κ p , and the corresponding number of power iterations (the algorithm terminates when the difference between two successive values is lower than 10⁻¹²).

From Table 4, we observe that the upper bounds of the condition numbers are sharp although there may be a factor O ( 10 2 ) between the exact absolute normwise condition number κ MSTLS abs and its upper bound κ ¯ MSTLS abs sometimes. κ p is always equal or very close to κ MSTLS abs .

In this paper, we are concerned with the matrix-scaled total least squares (MSTLS) problem with multiple right-hand sides. To our best knowledge, the condition numbers of the MSTLS problem have so far not been considered systematically. Based on this view, we focus on the normwise, mixed and componentwise condition numbers of the MSTLS problem under some mild conditions, respectively. Then the tight and computable upper bound estimates are provided. Numerical examples are given to illustrate the tightness of these bounds. In addition, large-scale problems are more interesting, we can make efforts on computational issues associated with these problems, possibly with additional structures (like sparseness) for the defining matrices. This will be a good research direction in the future.

The authors are grateful to the anonymous referees and the Editor for their detailed and helpful comments that led to a substantial improvement to the paper.

Longyan Li is supported by the Research and Training Program for College Students (No. A2020-17).

The authors declare no conflicts of interest regarding the publication of this paper.

Wang, Q., Li, L.Y. and Zhang, P.P. (2021) Perturbation Analysis for the Matrix-Scaled Total Least Squares Problem. Advances in Pure Mathematics, 11, 121-137. https://doi.org/10.4236/apm.2021.112008

Denote H = ( H 1 + H 2 ) R . This algorithm involves, however, the computation of the products of H by a vector f ∈ ℝ p ( m + n + d − 2 p ) and H T by a vector g ∈ ℝ n d . We describe now how to perform the two operations.

Let f = [ f 1 T f 2 T ] T ∈ ℝ p ( n + m + d − 2 p ) with f 1 ∈ ℝ p ( m − p ) , f 2 ∈ ℝ p ( n + d − p ) , F 1 = reshape ( f 1 , m − p , p ) and F 2 = reshape ( f 2 , n + d − p , p ) . It follows from Lemma 2.2 that

W γ ( V 11 T ) † = W γ V 11 − W γ V 12 V 22 † V 21 = W γ V 11 + X MSTLS V 21 ,

which together with Lemma 2.1 and the fact that W γ V 12 F V 22 = W γ V 12 ( I − V 22 † V 22 ) = W γ V 12 + X MSTLS V 22 gives

H f = ( H 1 + H 2 ) ( Σ 1 2 ⊗ I n + d − p − I p ⊗ ( Σ 2 T Σ 2 ) ) − 1 [ I p ⊗ Σ 2 T Σ 1 ⊗ I n + d − p ] [ vec ( F 1 ) vec ( F 2 ) ] = ( H 1 + H 2 ) ( Σ 1 2 ⊗ I n + d − p − I p ⊗ ( Σ 2 T Σ 2 ) ) − 1 vec ( Σ 2 T F 1 + F 2 Σ 1 ) = ( ( ( V 22 V 22 T ) − 1 V 21 ) ⊗ ( W γ V 12 F V 22 ) + ( ( V 22 † ) T ⊗ W γ ( V 11 T ) † ) Π ( n + d − p , p ) ) vec ( T ) = vec ( ( W γ V 12 + X MSTLS V 22 ) T V 21 T ( V 22 V 22 T ) − 1 + ( W γ V 11 + X MSTLS V 21 ) T T V 22 † ) , (29)

where T ∈ ℝ ( n + d − p ) × p with the i-th column

t i = T e i = ( σ i 2 I n + d − p − Σ 2 T Σ 2 ) − 1 ( Σ 2 T F 1 + F 2 Σ 1 ) e i = ( σ i 2 I n + d − p − Σ 2 T Σ 2 ) − 1 ( Σ 2 T F 1 ( : , i ) + σ i F 2 ( : , i ) ) .

Similarly, let g ∈ ℝ n d and G = reshape ( g , n , d ) . Then we have

H T g = R T ( ( ​ ( V 22 V 22 T ) − 1 V 21 ) ⊗ ( W γ V 12 F V 22 )   + ( ( V 22 † ) T ⊗ W γ ( V 11 T ) † ) Π ( n + d − p , p ) ) T vec ( G ) = [ I p ⊗ Σ 2 Σ 1 ⊗ I n + d − p ] ( Σ 1 2 ⊗ I n + d − p − I p ⊗ ( Σ 2 T Σ 2 ) ) − 1   × vec ( ( W γ V 12 + X MSTLS V 22 ) T G ( V 22 V 22 T ) − 1 V 21 + V 22 † G T ( W γ V 11 + X MSTLS V 21 ) ) = [ I p ⊗ Σ 2 Σ 1 ⊗ I n + d − p ] vec ( Z ) = [ vec ( Σ 2 Z ) vec ( Z Σ 1 ) ] , (30)

where Z ∈ ℝ ( n + d − p ) × p with the i-th column

z i = ( σ i 2 I n + d − p − Σ 2 T Σ 2 ) − 1 ( ( W γ V 12 + X MSTLS V 22 ) T G ( V 22 V 22 T ) − 1 V 21     + V 22 † G T ( W γ V 11 + X MSTLS V 21 ) ) e i = ( σ i 2 I n + d − p − Σ 2 T Σ 2 ) − 1 ( ( W γ V 12 + X MSTLS V 22 ) T G ( V 22 V 22 T ) − 1 V 21 ( : , i )     + V 22 † G T ( W γ V 11 ( : , i ) + X MSTLS V 21 ( : , i ) ) ) .

Using (29) and (30), we can now write in Algorithm 1 the iteration of the power method ( [27], p. 289) to compute the absolute and relative normwise condition numbers κ MSTLS abs and κ MSTLS rel of X MSTLS . In this algorithm we assume X MSTLS and the SVD of A γ are available.