Consider the computation of an approximate solution of the minimization problem:

where and throughout this paper, ‖   ⋅   ‖ denotes the Euclidean vector norm or the associated induced matrix norm. The singular values of the matrix A are assumed of different size closed to the origin. It follows that A is severely ill-conditioned and may be singular. Minimization problems (1) with a matrix of this kind often are referred to as discrete ill-posed problems. They arise, for example, from the discretization of linear ill-posed problem, such as Fredholm integral equations of the first kind with a smooth kernel [1]. The application background of discrete ill-posed problems is very extensive, such as signal processing, image denoising and deblurring [2]. We will for notational simplicity assume that m ≥ n ; however, the methods discussed also can be applied when m < n .

There are measurement or discretization errors e in b , here e ∈ ℝ m is called noise, and it is generally Gaussian white noise. Let b t r u e ∈ ℝ m denotes the (unknown) noise-free vector associated with b , i.e., b = b t r u e + e . We are interested in computing an approximation of the solution x ^ of the minimal Euclidean norm of the error-free least-squares problem:

often does not yield a meaningful approximation of x t r u e , because A † e causes a relatively large error. We would like to determine an approximate of x ^ by computing a suitable approximate solution of (1) [3].

For small to medium sized discrete ill-posed problems, the Tikhonov regularization method is most commonly used for numerical solution [4]. This method replaces (1) by a penalized least-squares problem

where L ∈ ℝ p × n ( p ≤ n ) is the regularization matrix, and the scalar μ > 0 is the regularization parameter. The normal equations associated with (3) are given by ( A T A + μ − 2 L T L ) x = A T b . The matrix L is assumed to satisfy

where N ( M ) denotes the null space of matrix M , then (3) has a unique solution

Throughout this paper A T denotes the transpose of matrix A . The value of μ derermines how sensitive x μ is to the error e and how close x μ is to x ^ , and Hansen for discussions on Tiknonov regularization [5]. Both the choice of the regularization parameter μ and the regularization matrix L are crucial, because they affect the approximation degree of the solution to (3) with the true solution x t r u e and the sensitivity of the solution of (3) to noise. L is generally selected as an orthogonal matrix or a banded matrix with known null space, such as the identity matrix, first-order or second-order derivative operator matrices. When L is the identity matrix, the Tikhonov minimization problem (3) is said to be in standardform, otherwise it is said to be in generalform. When the magnitude of the noise is known, the discrepancy principle is used to determine μ ; when the magnitude of the noise is unknown, the L-curve criterion and Generalized Cross Validation (GCV) are used to determine μ , see, e.g., [6]-[9]. There are also other methods, such as the vector extrapolation method and the three-point interpolation zero-finding method [10]-[12].

Regularization methods and Singular Value Decomposition methods are often used for numerical solution of the system (1). Hansen et al. proposed the Truncated Singular Value Decomposition (TSVD) in [5][13]. This method based on the singular value decomposition of matrix A , find an optimal rank- k matrix A k to approximate A , so that the approximate problem A k x = b approximates (1). The truncated index k is determined by the discrete Picard condition [14]. Morigi et al. proposed the Truncated Projection SVD method (TPSVD) in [15]. This method applied orthogonal projection to matrix A and the right-hand side vector b to obtain a projection problem, and then apply TSVD to solve it. Reichel proposed the Modified Truncated Singular Value Decomposition (MTSVD) in [16]. This method modify the truncation index and some singular values, so that the coefficient matrices determined by TSVD and MTSVD have the same spectral condition number, but MTSVD can obtain a better approximate solution. Zhongxiao Jia [17] proposed the Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for large scale discrete ill-posed problems with general-form regularization.

In this paper, for small-to-medium sized discrete ill-posed problems, we propose a new Modified Truncated Singular Value Decomposition (NMTSVD) base on the MTSVD method. This method presents three approaches to select truncation index and three new partial singular value correction methods, leading to three approximate matrices of the coefficient matrix A . And the relationships of these approximate matrices are established in terms of the spectral condition number. Compared with TSVD and MTSVD, NMTSVD can obtain better approximate solutions.

For the large-scale nostable iterative Tikhonov regularization problem [18].

where x 0 ∈ ℝ n is the initial approximate solution, generally taken as x 0 = 0 . Only when A and L satisfy (4) can be ensured the system (5) to have the unique solution. The regularization parameter μ − 2 has advantages over μ . The selection of the regularization matrix and parameter is crucial, leading to affect the approximation accuracy between the approximate solution of (5) and the exact solution. The unstable iterative Tikhonov regularization method yields solutions with higher approximation accuracy [7][19]-[21]. The normal equations associated with (5) are given by

The computed approximation solution x μ k , lives in the series of low-rank k -dimensional Krylov subspace

The Golub-Kahan bidiagonalization (GKB) is popularly applied to reduce large-scale unstable discrete ill-posed problems to small dimension minimization problems. However, during the solution process of the iterative method, a semi-convergence phenomenon may occur in the approximate solution, i.e., the convergence effect of the approximate solution initially improves, but when the number of iterations exceeds a certain value, the approximate solution tends to diverge [22][23].

The columns of orthogonal matrices U k , V k of the k steps of Golub-Kahan bidiagonalization to A tend to lose orthogonality, therefore necessary to apply the reorthogonalization strategy of the columns of V k to preserve the orthogonality of the orthogonal matrices and convergence of the singular value. Many researchers concern about the reorthogonalization strategy [24][25] for deeply studying the Golub-Kahan bidiagonal matrix factorization of A . Paige [26] pointed out that the lose of orthogonality in lanczos reductions is structured in the sense that it is coincident with the convergence of approximate eigenvalues and eigenvectors (calls Ritz and vectors). Parlett and Scott [27] used this observation to develop partial reorthogonalization procedures, and gave a good summary of the surrounding issues. Barlow supported three reorthogonalization strategies: complete reorthogonalization, selective reorthogonalization and parameterrized reorthogonalization, see [28] for details. Ramlau [29] proposed the error estimates for Golub-Kahan-digiagonalization for linear ill-posed problems. Reichel [30] proposed the iterated Golub-Kahan-Tikhonov method for large discrete linear ill-posed problems.

This paper combines the GKB method with the reorthogonalization method and truncated the dimension of Krylov subspace, referred to as the Truncated Reorthogonalized Golub-Kahan Bidiagonalization method (RGKB). Compared with GKB, RGKB yields a better approximate solution.

This paper is organized as follows. In Section 2, we propose a novel Modified Truncated Singular Value Decomposition (NMTSVD) methods based on MTSVD for small-to-medium sized discrete ill-posed problems, presenting the filter factor representations for their approximate solutions. In Section 3, introduce the Truncated Reorthogonalized Golub-Kahan Bidiagonalization (RGKB) method for large-scale discrete ill-posed problems, and the RGKB algorithm is presented. In Section 4, present numerical experiments demonstrating the feasibility and effectiveness of NMTSVD and RGKB.

In this paper, we first propose the Gram-Schmidt reorthogonalization process to reorthogonalize V k , which is summarized as function 1; U k is still calculated using the GKB recurrence formula (14), and the orthogonality of U k will not be lost, which is summarized as function 2. Second, in order not to increase too much computational complexity, the iteration truncate the dimension k ¯ of the Krylov subspace, which is summarized as Algorithm 1. Finally, Algorithm 2 for the truncated reorthogonalized Golub-Kahan bidiagonalization method (RGKB) is presented, and the regularization parameter μ is obtained by minimizing the GCV function.

The recursion formulas for the GKB decomposition of A with the initial vector b are given by,

The k steps of Golub-Kahan bidiagonalization to A with the initial vector b givens the decompositions,

where the matrices U k = [ u 1 , u 2 , ⋅ ⋅ ⋅ , u k ] ∈ ℝ m × k , U k + 1 = [ U k , u k + 1 ] ∈ ℝ m × ( k + 1 ) , and V k = [ v 1 , v 2 , ⋅ ⋅ ⋅ , v k ] ∈ ℝ n × k have orthogonal columns.

is lower bidiagonal matrix, and C ˜ k = [ C k T , β k + 1 e k ] T . Here,

e j = [ 0 , ⋅ ⋅ ⋅ , 0 , 1 , 0 , ⋅ ⋅ ⋅ , 0 ] T is the j -th axis vector. The columns of the matrix V k span the Krylov subspace (7).

Introduce the QR factorization of L V k ,

where Q k ∈ ℝ p × k has orthogonal columns and R k ∈ ℝ k × k is upper triangular.

From the GKB decomposition (14) of A and the QR decomposition (15) of L V k , we have

The influence matrix is given by A ♯ = ( A T A + μ k − 2 L T L ) − 1 A T . Let x μ = V k y k , where y k ∈ ℝ k , then the Generalized Cross-Validation (GCV) function is defined as follows:

where the numerator part is

The matrix part in the denominator is:

where r k = b − A x k , r k − 1 = b − A x k − 1 , and t o l is a user-defined very small error margin.

Left-multiplying both sides of equation (6) by the matrix V k T , we have

Since A and L satisfy (4), the matrix [ A V k , L V k ] is of full rank. For any 0 < μ k < ∞ , the matrix on the left-hand side of (16) is nonsingular. Therefore,

has a unique solution y k .

The termination condition for the iteration is:

As the number of iteration steps k increases, the orthogonal matrices U k and V k in (14) tend to lose their orthogonality. To ensure the orthogonality of U k and V k , the Gram-Schmidt reorthogonalization process is used to reorthogonalize V k . U k is still calculated using the recursive formula (14), and its orthogonality will not be lost. For the detailed proof of the reorthogonalization process, refer to Barlow’s paper [28].

Let r k = A T u k + 1 − β k + 1 v k , and the Gram-Schmidt reorthogonalization process is summarized into the following subfunction GS_reorth.

function 1: [ v k + 1 , h , α k + 1 ] = GS_reorth ( V k , r k , s e t z e r o )

if ‖ r k 1 ‖ 2 ≥ 0.8 ‖ r k ‖ 2

else

if ‖ r k 2 ‖ 2 ≥ 0.8 ‖ r k 1 ‖ 2

else

Find e J such that

if setzero

else

end

end

end

end GS_reorth

First, the GKB method with the GS process is presented and summarized as the subfunction GKB_reorth.

function 2 [ U , C , V ] = GKB_reorth ( A , b , k , s e t z e r o )

for j = 1 , ⋯ , k

end

end GKB_reorth.

Next, to ensure the orthogonality of the orthogonal matrices U k and V k without adding excessive computational complexity, this paper truncates the iteration process at subscript k ¯ , meaning the dimension of the solution space is k ¯ . This is summarized as Algorithm 0. Finally, the process of the RGKB method proposed in this paper is summarized as Algorithm 0.

RGKB method proposed in this paper is summarized as Algorithm 2.

Ultimately, only the k ¯ -dimensional solution space V k ¯ is considered. Due to the semi-convergence property of iterative methods for large-scale discrete ill-posed problems, the convergence within the k ¯ -dimensional solution space is excellent, while the convergence beyond this space deteriorates. This phenomenon will be demonstrated in the experimental section of Section 4.2.

This chapter presents two experiments to verify the effectiveness of the proposed NMTSVD methods. The experiments are conducted using MATLAB R2024b on an INTEL(R) Core(TM) i3-2310M CPU (2.1GHz) with 2 GB RAM.

We use examples from the MATLAB package Regularization Tools [31]. The error e in the vector b is Gaussian white noise, with the noise level defined as:

The regularization matrix is,

The measure of approximation between the approximate solution x and the true solution x t r u e is:

The free parameter t o l in the termination condition (18) is set to 1 × 10⁻⁶.

Sincere thanks to the members of JAMP for their professional performance, and special thanks to managing editorHellen XU for a rare attitude of high quality.

This work was supported by the Gansu Province Innovation Fund for College Teachers (Grant No. 2024A-173).