Throughout this paper, one studies a solution method for the system of linear equations which occurs throughout the literature [1] - [11] .

A x = b (1)

where A = ( a i j ) ∈ R n × n is non-singular matrix with unit diagonal b and x ∈ R n with x being an unknown vector in R n .

Splitting the matrix A into

A = I − L − U (2)

where I is the identity matrix, L and U are the strictly lower and strictly upper triangular parts of the matrix A, respectively; see [8] and [9] , the successive overrelaxation iteration formula for the SOR iteration method for an optimal relaxation factor ω is as follows:

x k + 1 = ( I − ω L ) − 1 ( ( 1 − ω ) I + ω U ) x k + ω b . (3)

While the following 2-step scheme:

( I − ω L ) y k = ( ( 1 − ω ) I + ω U ) x k + ω b , ( I − ω U ) x k + 1 = ( ( I − ω ) I + ω L ) y k + ω b (4)

for any initial guess x 0 defines the iteration formula for the SSOR iteration method with relaxation factor ω > 0 , [12] and [10] . The study of allowable ω for convergence of (4) appears throughout the literature [2] and [5] .

Define the iteration matrix for the SOR iteration method as

Δ ( ω ) : = ( I − ω L ) − 1 ( ( 1 − ω ) I + ω U ) , (5)

and the iteration matrix for the SSOR iteration method as

Γ ( ω ) : = ( I − ω U ) − 1 ( ( 1 − ω ) I + ω L ) ( I − ω L ) − 1 ( ( 1 − ω ) I + ω U ) . (6)

The required two step iteration is a drawback of the SSOR iteration method, when compared to SOR iteration method. However, for a symmetric matrix A, the iteration matrix has real spectrum, and the spectral radius of Γ ( ω ) i.e. ρ ( Γ ( ω ) ) , is equal to the square of a norm of Δ ( ω ) ( [9] : pg. 237). This paper introduces a variation of the SSOR iteration method based on the work of Kellogg [13] and is similar to work that appears in [7] for the HSS algorithm. This new algorithm requires approximately the amount of work per iteration as the SOR iteration method, and uses an iteration matrix which is similar to the SSOR iteration method matrix with equal spectral radius. This iteration scheme will be entitled the Kellogg-type SSOR iteration method, and its iteration matrix will be denoted by the KSSOR iteration matrix.

Letting the terms in Γ ( ω ) be somewhat permuted, one defines

Ψ ( ω ) : = ( I − ω L ) − 1 ( ( 1 − ω ) I + ω L ) ( I − ω U ) − 1 ( ( 1 − ω ) I + ω U ) , Θ ( ω ) : = ( I − ω U ) − 1 ( ( 1 − ω ) I + ω U ) ( I − ω L ) − 1 ( ( 1 − ω ) I + ω L ) . (7)

If A is a symmetric matrix then Ψ ( ω ) and Θ ( ω ) are symmetric matrices and in this case, the SSOR iteration matrix has real spectrum. A relationship between the spectral radii of the three matrices Γ ( ω ) , Ψ ( ω ) and Θ ( ω ) follows.

Lemma 1. ρ ( Γ ( ω ) ) = ρ ( Ψ ( ω ) ) = ρ ( Θ ( ω ) ) , for all ω > 0 .

Proof. Since ( I − ω U ) Γ ( ω ) ( I − ω U ) − 1 = ( ( 1 − ω ) I + ω L ) ( I − ω L ) − 1 ( ( 1 − ω ) I + ω U ) ( I − ω U ) − 1 = Ψ ( ω ) , then Γ ( ω ) is similar to Ψ ( ω ) . Now Θ ( ω ) and Ψ ( ω ) have the same spectral radius from the well known product theorem [14] , i.e. ρ ( A B ) = ρ ( B A ) for any A and B. Hence, ρ ( Γ ( ω ) ) , ρ ( Ψ ( ω ) ) and ρ ( Θ ( ω ) ) are equal.

Lemma 1 suggests using a different ordering for the SSOR iteration scheme.

Assuming that the SSOR iterative method converges for ω > 0 , i.e. ρ ( Γ ( ω ) ) < 1 with ω a positive acceleration factor, then the Kellogg-type SSOR iteration method is written as:

( I − ω L ) y k = ( ( 1 − ω ) I + ω L ) x k + ω b 1 , ( I − ω U ) x k + 1 = ( ( 1 − ω ) I + ω U ) y k + ω b 2 (8)

where b = b 1 + b 2 . Now for z = y k + x k + 1 in our trial solution, one tests ‖ A z − b ‖ where ‖   ⋅   ‖ is the 2-norm. If this quantity fails to satisfy our convergence criteria, one assigns x k + 1 to x k , and reiterates.

Upon eliminating y k and the x k , from (8) one has

x k + 1 = Θ ( ω ) x k + ( I − ω L ) − 1 ( ( 1 − ω ) I + ω L ) ( I − ω U ) − 1 ω b 1 + ( I − ω U ) − 1 ω b 2 and y k + 1 = Ψ ( ω ) y k + ( I − ω U ) − 1 ( ( 1 − ω ) I + ω U ) ( I − ω L ) − 1 ω b 2 + ( I − ω L ) − 1 ω b 1 . (9)

This decouples the original system (4).

The following lemma will be used to show the convergence of the Kellogg-type SSOR iteration method to a unique solution to the system of equations A x = b .

Lemma 2. For any c 1 ,   c 2 then there exists unique y ,   x satisfying

x = ( I − ω L ) − 1 ( ( 1 − ω ) I + ω L ) y + c 1 , y = ( I − ω U ) − 1 ( ( 1 − ω ) I + ω U ) x + c 2 . (10)

Furthemore,

y = Ψ ( ω ) y + ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) c 2 + c 1 , x = Θ ( ω ) x + ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) c 1 + c 2 . (11)

Proof. The system of equations (10) may be represented by

A ˜ [ x y ] = [ c 1 c 2 ] ,

where

A ˜ = [ I − ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) − ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) I ] = [ I 0 − ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) I ] [ I 0 0 I − Θ ( ω ) ]     ∗ [ I − ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) 0 I ] .

The det of A ˜ is non-zero, completing the proof of the first part. The second part follows immediately.

The following theorem proves that the sequences { x k } and { y k } converge to x and y, respectively, and x + y is the solution to A x = b .

Theorem 1. Let x k and y k be defined by (9) Then (i) lim k → ∞ x k = x , lim k → ∞ y k = y , and (ii) A ( x + y ) = b .

Proof. Setting e n = x − x n , c 1 = ( I − ω L ) − 1 ω b 1 and c 2 = ( I − ω U ) − 1 ω b 2 . Then y − y n = ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) x + c 1 − ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) x n − c 1 = ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) x − ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) x n . Since x = ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) y + c 2 and x n = ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) y n − 1 + c 2 , then ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) x − ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) x n becomes

Θ ( ω ) y + ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) c 2 − Θ ( ω ) x n − ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) c 2

In which case, e n + 1 = Θ ( ω ) e n and since ρ ( Θ ( ω ) ) < 1 , then lim k → ∞ x k = x . In a similar fashion, lim k → ∞ y k = y .

To show that A ( x + y ) = b , set c 1 = ( I − ω L ) − 1 ω b 1 and c 2 = ( I − ω U ) − 1 ω b 2 by (i) and Lemma 2, the vector [ x y ] satisfies

y = ( I − ω L ) − 1 ( ( 1 − ω ) I + ω L ) x + c 1 x = ( I − ω U ) − 1 ( ( 1 − ω ) I + ω U ) y + c 2 . (12)

Rewriting this as

( I − ω L ) y = ( ( 1 − ω ) I + ω L ) x + ω b 1 ( I − ω U ) x = ( ( 1 − ω ) I + ω U ) y + ω b 2 , (13)

and then adding them together, canceling, and dividing by ω . This simplifies to

( x + y ) − L ( x + y ) − U ( x + y ) = ( I − L − U ) ( x + y ) = b 1 + b 2 as promised.

A flowchart for the Kellog-type SSOR iteration method follows, see Figure 1.

For strictly lower and strictly upper triangular parts of an n × n matrix, the following holds.

Lemma 3. For L and U the strictly lower and strictly upper triangular parts of the n × n matrix A then

( I − ω L ) − 1 ( ( 1 − ω ) I + ω L )     and     ( I − ω U ) − 1 ( ( 1 − ω ) I + ω U ) (14)

are lower triangular and upper triangular matrices respectively.

Proof. Since L n is equal to the zero matrix and ρ ( ω L ) < 1 then ( I − ω L ) − 1 exists and is given by ∑ i = 0 n − 1 ( ω L ) i .

Upon multiplying by ( ( 1 − ω ) I + ω L ) gives the result that ( I − ω L ) − 1 ( ( 1 − ω ) I + ω L ) is a lower triangular matrix.

Similarly, ( I − ω U ) − 1 ( ( 1 − ω ) I + ω U ) is upper triangular.

One of the main drawbacks of the SSOR iteration method when compared to SOR iteration method is its two step iteration. Some benefits of SSOR iteration method for a symmetric matrix A, is Γ ( ω ) will have positive real spectrum and that ρ ( Γ ( ω ) ) is equal to the square of a norm of Δ ( ω ) . See [9] pg. 237. In the KSSOR iteration method, if one takes b 1 = b and b 2 = 0 , then each step is a triangular matrix (one lower, in the first step of the iteration, while upper in the second step of the iteration) times a vector operation. Since the amount of

work done by KSSOR iteration may be considered less than the SSOR iteration method. Furthermore, since the spectral radius of the SSOR iteration matrix and the KSSOR iteration matrix are equal, their iteration counts should be fairly equal. However, in most instances the inverse would not be taken.

A Cyclic Reduction Algorithm, similar to [3] , and [8] pg. 170, is as follows.

Set A ˜ = [ I − ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) − ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) I ] one sets K = I − A ˜ = [ 0 ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) 0 ] and then K 2 = [ Θ ( ω ) 0 0 Ψ ( ω ) ] .

Lemma 4. If ρ ( Γ ( ω ) ) < 1 then ρ ( K 2 ) < 1 and ρ ( K ) < 1 .

Proof. Since σ ( K 2 ) = σ ( Θ ( ω ) ) ∪ σ ( Ψ ( ω ) ) = σ ( Θ ( ω ) ) , where the last equality follows from the similarity of Θ ( ω ) and Ψ ( ω ) then ρ ( K 2 ) < 1 . By the spectral mapping theorem ± λ ∈ σ ( K ) if and only if λ ∈ σ ( K 2 ) ; i.e. determinant ( A 2 − λ I ) = determinant ( A + λ I ) determinant ( A − λ I ) . hence, ρ ( K ) < 1 , see [14] . This completes the proof of the lemma.

K 2 defines the following uncoupled iteration scheme.

x m + 2 = Θ ( ω ) x m + k 1 y m + 2 = Ψ ( ω ) y m + k 2 (15)

where k 1 = ( I − ω L ) − 1 ( ( I − ω ) I + ω L ) ( I − ω U ) − 1 b 2 + ( I − ω L ) − 1 b 1 and k 2 = ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) ( I − ω L ) − 1 b 1 + ( I − ω U ) − 1 b 2 .

We now focus our attention on the solution of this reduced equation x m + 2 = Θ ( ω ) x m + k 1 .

Since ρ ( Θ ( ω ) ) < 1 , then the Cyclic Reduction Scheme x m converges to x and having found the vector x in (10), form the vector y = ( I − ω U ) − 1 ( ( I − ω ) I + ω U ) x + ( I − ω U ) − 1 b 2 and from which x + y is a solution.

A flowchart for the Cyclic Reduction iteration method follows, see Figure 2.

In the following examples, a solution vector x is constructed as

x ( i ) = i N sin ( i ∗ π 6 ) for i = 1 , ⋯ , N and an initial guess is x 0 = [ 1,1, ⋯ ,1 ] T .

The error in each of our schemes will be measured by the 2-norm ‖ A y − b ‖ . where y is the resulting approximate answer and our desired error tolerance will be 10⁻⁶. Our table includes the ‖ x − y ‖ i.e. the actual 2-norm difference between the true solution, x, and our approximate solution y, the iteration count, and the elapsed time in seconds for the work of the loop doing the iteration. ω is found through an iterative search, all decimals are rounded to two places and in the KSSOR iteration method and the Cyclic Reduction Method, b 1 = b and b 2 = 0 .

Example 1. Consider the system of linear equations with the n × n symmetric matrix which appears in [4] Set P to be the m × m matrix

P = [ 2 − 1 0 0 0 0 0 0 − 1 2 − 1 0 0 0 0 0 0 − 1 2 − 1 0 0 ... 0 . . . . . . 0 0 0 .. − 1 2 − 1 0 0 0 0 .. 0 − 1 2 ]

and T = ( I ⊗ P + P ⊗ I ) − ( P ⊗ P ) / 6 , and A = D − 1 ∗ T where D is the diagonal of T. The resulting matrix A, is then n × n where n = m 2 . For m = 32 , the n × n matrix is 1024 × 1024 and Table 1 summarizes our results.

Example 2. Consider the system of linear equations with 1024 × 1024 coefficient matrix which appears in [11] .

T = [ 1 1 0 0 0 0 0 0 − 1 3 2 0 0 0 0 0 0 − 1 5 3 0 0 ... 0 . . . . . . 0 0 0 .. − 1 2 ∗ N − 3 N − 1 0 0 0 0 .. 0 − 1 2 ∗ N − 1 ]

Then A becomes D − 1 ∗ T where D is the diagonal of T. Table 2 summarizes our results.

Example 3 Consider the system of linear equations with n × n coefficient matrix

T = [ B E − E ′ 0.5 ∗ I ]

where B = [ I ⊗ C + C ⊗ I 0 0 I ⊗ C + C ⊗ I ] , E = [ I ⊗ F F ⊗ I ] .

Where C = tridiag ( − 1,2, − 1 ) , F = h ∗ tridiag ( − 1,1,0 ) are m × m matrices with h = 1 m + 1 resulting in n = 3 m 2 . Then A becomes D − 1 ∗ T where D is the diagonal of T and for m = 20 is a 1200 × 1200 matrix. Table 3 summarizes our results.

Since the spectral radius of all iteration matrices is the same in all the iteration schemes, the convergence to a solution of the system of linear equations has about the same number of iterations for all the iteration convergence schemes considered in this paper.

The KSSOR algorithm requires an upper and lower triangular solution to a linear system which may be more accurate than the SSOR algorithm which on the surface requires full matrix solves at each step.

The authors declare no conflicts of interest regarding the publication of this paper.

Smotzer, T. and Buoni, J. (2024) Another SSOR Iteration Method. American Journal of Computational Mathematics, 14, 248-256. https://doi.org/10.4236/ajcm.2024.142009