A method for solving systems of linear equations is presented based on direct decomposition of the coefficient matrix using the form LAX = LB = B’ . Elements of the reducing lower triangular matrix L can be determined using either row wise or column wise operations and are demonstrated to be sums of permutation products of the Gauss pivot row multipliers. These sums of permutation products can be constructed using a tree structure that can be easily memorized or alternatively computed using matrix products. The method requires only storage of the L matrix which is half in size compared to storage of the elements in the LU decomposition. Equivalence of the proposed method with both the Gauss elimination and LU decomposition is also shown in this paper.
Systems of linear equations or equations linearized for iterative solutions arise in many science and engineering problems [
Gaussian elimination is a systematic way of reducing systems of linear equations into a triangularised matrix through addition of the independent equations [
The LU decomposition was developed by Alan Turing as an alternative way carrying out Gaussian elimination through factorization of the coefficient matrix into a product of upper and lower triangular matrices, namely, A = LU [
The Cholesky factorization works for symmetric positive definite matrices. The coefficient matrix in the system of equation AX = B is factorized into A = LLT where L is the lower triangular matrix with its transpose LT being an upper triangular matrix. The solution involves solving successively for LY = B and LTX = Y [
The QR decomposition transforms the system of equation AX = B into triangular system RX = QTb where A = QR. The matrix Q is orthogonal (QQT = I) and R is an upper triangular matrix [
The method proposed in this paper is based on reducing the coefficient matrix A in the system of linear equations AX = B using a single lower triangular reducing matrix L. The original coefficient matrix A is transformed into an upper triangular matrix U that allows solution through back substitution as is usual with both LU decomposition as well as Gauss elimination methods. For the original system of n by n linear equations given as:
a 11 x 1 + a 12 x 2 + a 13 x 3 + ⋯ + a 1 n x n = b 1 a 21 x 1 + a 22 x 2 + a 23 x 3 + ⋯ + a 2 n x n = b 2 ⋮ a n 1 x 1 + a n 2 x 2 + a n 3 x 3 + ⋯ + a n n x n = b n (1)
The matrix representation of Equation (1) will be:
A X = B (2)
where A is the coefficient matrix having the elements aij of the original equations and B is the right hand side column vector containing the elements b 1 , b 2 , ⋯ , b n .
The proposed method establishes a solution that transforms both the coefficient matrix A and the right hand side column vector B as follows:
L A X = U X = L B = B ′ (3)
In other words the coefficient matrix and the right hand side column vector B are transformed through the equations:
L A = U and L B = B ′ (4)
The procedure, therefore, essentially centers on determining the lower triangular matrix L that reduces the coefficient matrix A to an upper triangular matrix U. Let this matrix L be given through its elements lij so that:
L = [ 1 0 0 0 0 0 l 21 1 0 0 0 0 l 31 l 32 1 0 0 0 l 41 l 42 l 43 1 0 0 ⋮ ⋮ ⋮ ⋱ 1 0 l n 1 l n 2 l n 3 ⋯ ⋯ 1 ] (5)
The operation LA = U will reduce the coefficient matrix A in to an upper triangular matrix U given by:
U = [ u 11 u 12 u 13 u 14 ⋯ u 1 n 0 u 22 u 23 u 24 ⋯ u 2 n 0 0 u 33 u 34 ⋯ u 3 n 0 0 0 u 44 ⋯ u 4 n 0 0 0 0 ⋱ ⋮ 0 0 0 0 ⋯ u n n ] (6)
However, this proposed method does not need storage of the U matrix as only the L matrix needs to be determined and used to reduce both the A matrix and the right hand side column vector B. This is easily seen through the matrix operation involving the reducing matrix L only, namely,
L A X = L B = B ′ (7)
In this method, the lij elements will be written in terms of the Gauss pivot row multipliers mij of the Gauss elimination, and, as will be shown shortly, the lij elements are the sum of the permutation products of the mij multipliers assembled into a tree like structure for easy memorization. The elements lij will not remain constant during the reduction process as is normally the case with Gauss elimination or LU decomposition, but change as the reduction of A to U matrix progresses column wise or row wise as new members of the Gauss pivot row multipliers are added to the element lij.
Unlike the Gauss method which is restricted to column wise operation, in this method it is also possible to proceed row wise. In fact the row wise procedure will be followed to derive the lij elements.
Starting with row 2 of the lower triangular L matrix,, the only unknown is l21 and in terms of the Gauss elimination pivot row multipliers mij, the pivot operation to educe u21 to zero is given as:
m 21 a 11 + a 21 = 0 so that m 21 = − a 21 / a 11 (8)
For row 3, l31 is determined through the pivot element a11 so that:
m 31 a 11 + a 31 = 0 so that m 31 = − a 31 / a 11 (9)
For row 3 again, the remaining element l32 is determined through the pivot element a ′ 22 which is modified from the original value a22 because of the earlier reduction operation on row 2. Hence the reduction to u32 = 0 is given by:
m 32 ( m 21 a 12 + a 22 ) + ( m 31 a 12 + a 32 ) = 0 (10)
Collating the pivot row multipliers m with respect to the coefficient matrix A elements, namely, aij, Equation (10) becomes:
( m 31 + m 32 m 21 ) a 12 + m 32 a 22 + a 32 = 0 (11)
Likewise the lij elements for row 4 are determined as follows:
For ( u 41 = 0 ),
m 41 a 11 + a 41 = 0 (12)
For ( u 42 = 0 ),
( m 41 + m 42 m 21 ) a 12 + m 42 a 22 + a 42 = 0 (13)
For ( u 43 = 0 ),
( m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ) a 13 + ( m 42 + m 43 m 32 ) a 23 + a 43 = 0 (14)
For a 4 × 4 L matrix, summarizing the lij elements, expressed in terms of the Gauss pivot row multipliers m shown above, will give the L matrix shown in Equation (15).
L = [ 1 0 0 0 m 21 1 0 0 m 31 + m 32 m 21 m 32 1 0 m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 m 42 + m 43 m 32 m 43 1 ] (15)
It is easy to show that the m terms in the L matrix in Equation (15) form permutation products where by the number of terms correspond to coefficients of the binomial series expansion. For any element lij of the L matrix, the number of m-product terms is given by:
N m ( i , j ) = 2 i − j − 1 (16)
The power of binomial expansion K ( i , j ) is given by;
K ( i , j ) = i − j − 1 (17)
For example, for l41,
N m ( 4 , 1 ) = 2 4 − 1 − 1 = 2 2 = 4
This corresponds to the binomial expansion of power K ( 4 , 1 ) = 2 , i.e., { 1 , 2 , 1 } .
The permutation products l41 as shown in the L matrix are:
l 41 = { m 41 , m 42 m 21 + m 43 m 31 , m 43 m 32 m 21 } (18)
For l51 similarly,
N m ( 5 , 1 ) = 2 5 − 1 − 1 = 2 3 = 8
This corresponds to the binomial expansion of power K ( 5 , 1 ) = 3 , i.e., { 1 , 3 , 3 , 1 } .
The permutation m-products for l51 of the L matrix are, therefore,
l 51 = { m 51 , m 54 m 41 + m 53 m 31 + m 52 m 21 , m 54 m 42 m 21 + m 53 m 32 m 21 + m 54 m 43 m 31 , m 54 m 43 m 32 m 21 } (19)
It is easy to enumerate the m-permutation products of lij as these products can be arranged in a tree-like structure. Taking the example of elements of l51 for example, the tree structure shown in
For the element lij of the lower triangular matrix L, with the number of m products Nm corresponding to the binomial coefficients of power K(i,j), the binomial coefficients Nm(r) for r = 0 , 1 , 2 , ⋯ , K is given by:
N m ( r ) = C r K = K ! ( K − r ) ! r ! (20)
For example for l41 with K = 4 − 1 − 1 = 2 and K ( r ) = { 0 , 1 , 2 } ;
N m ( 0 ) = C 0 2 = 2 ! ( 2 − 0 ) ! 0 ! = 1
N m ( 1 ) = C 1 2 = 2 ! ( 2 − 1 ) ! 1 ! = 2
N m ( 2 ) = C 2 2 = 2 ! ( 2 − 2 ) ! 2 ! = 1
Hence, N m = 1 + 2 + 1 = 4 .
Similarly for l51 with K ( 5 , 1 ) = 5 − 1 − 1 = 3 and K ( r ) = { 0 , 1 , 2 , 3 } ;
N m ( 0 ) = C 0 3 = 3 ! ( 3 − 0 ) ! 0 ! = 1
N m ( 1 ) = C 1 3 = 3 ! ( 3 − 1 ) ! 1 ! = 3
N m ( 2 ) = C 2 3 = 3 ! ( 3 − 2 ) ! 2 ! = 3
N m ( 3 ) = C 3 3 = 3 ! ( 3 − 3 ) ! 3 ! = 1
Once the Nm(r) values are determined corresponding to the binomial coefficients the sum of permutation products are calculated as follows:
As an example for l51 with K = { 0 , 1 , 2 , 3 } and taking K = 2 which contain 3 terms, the sum of permutation products M i j ( K = 2 ) is given by:
M i j ( K = 2 ) = ∑ S = i − 1 K + 1 ∑ P = S − 1 j + 1 m i S m S P m p j (21)
M 51 ( K = 2 ) = ∑ S = 4 3 ∑ P = S − 1 2 m 5 S m S P m p 1 (22)
M 51 ( K = 2 ) = m 54 m 43 m 31 + m 54 m 42 m 21 + m 53 m 32 m 21 (23)
In general for any element lij, the mij sum of products can be calculated using the formula:
M i j ( K ) = ∑ S = i − 1 K + 1 ∑ P = S − 1 K ∑ Q = p − 1 K − 1 ⋯ ∑ t = K + 1 j + 1 m i S m S P m p q ⋯ m t j (24)
Finally, the element lij is computed by summing the Mij sum of products as follows:
l i j = ∑ W = 0 W = K M i j ( W ) (25)
The computation of elements of the lower triangular matrix L can be easily carried out using matrix multiplication. For any element lij the matrix multiplication takes the following form:
l i j = [ m i j m i j + 1 m i j + 2 ⋯ m i i − 1 ] ∗ [ 1 l j + 1 j l j + 2 j l j + 3 j ⋮ l i − 1 j ] (26)
Equation (26) shows the lij can be determined from already determined previous values of lkj where j + 1 < k < i and the Gauss pivot row multipliers m i j , m i j + 1 , m i j + 2 , ⋯ , m i i − 1 .
Equation 26 can be summarized in the general matrix product form as follows. Considering the lower triangular matrix L of Equation (5) again;
L = [ 1 0 0 0 0 0 l 21 1 0 0 0 0 l 31 l 32 1 0 0 0 l 41 l 42 l 43 1 0 0 ⋮ ⋮ ⋮ ⋱ 1 0 l n 1 l n 2 l n 3 ⋯ ⋯ 1 ] (5)
The negative of the corresponding Gauss pivot row multipliers mrs that are already determined at this stage are given by the matrix form LLU;
L L U = [ 1 0 0 0 0 0 − m 21 1 0 0 0 0 − m 31 − m 32 1 0 0 0 − m 41 − m 42 − m 43 1 0 0 ⋮ ⋮ ⋮ ⋱ 1 0 − m n 1 − m n 2 − m n 3 ⋯ ⋯ 1 ] (27)
The matrix LLU is simply the L matrix of the LU decomposition method. This can be verified as follows:
To avoid confusion, let traditional LU decomposition method have its L matrix relabelled LLU to make it different from the L matrix of the proposed direct decomposition procedure.
From the relationship A = LLUU as well as LA = U, it follows that:
A = L − 1 U = L L U U (28)
It follows then that:
L − 1 = L L U or
L = L L U − 1 or
L ∗ L L U − 1 = I (29)
in which I is the identity matrix. Therefore the L matrix is simply the inverse of the L matrix LLU of the LU decomposition method.
The L matrix elements as shown in Equation (15) can be can be reproduced from the matrix equation
L L U ∗ L = [ 1 0 0 0 0 0 − m 21 1 0 0 0 0 − m 31 − m 32 1 0 0 0 − m 41 − m 42 − m 43 1 0 0 ⋮ ⋮ ⋮ ⋱ 1 0 − m n 1 − m n 2 − m n 3 ⋯ ⋯ 1 ] [ 1 0 0 0 0 0 l 21 1 0 0 0 0 l 31 l 32 1 0 0 0 l 41 l 42 l 43 1 0 0 ⋮ ⋮ ⋮ ⋱ 1 0 l n 1 l n 2 l n 3 ⋯ ⋯ 1 ] = I (30)
This computation will be illustrated for the 4 by 4 matrix of L shown in Equation 5 and later for the example of the 4 by 4 system of linear equations solved in the section that follows. Starting with the element l21 the matrix form of Equation (30) will take the form:
l 21 = [ m 21 ] ∗ [ 1 ] = 1 (31)
For element l31:
l 31 = [ m 31 m 32 ] ∗ [ 1 l 21 ] = [ m 31 m 32 ] ∗ [ 1 m 21 ] = m 31 + m 32 m 21 (32)
For element l32:
l 32 = [ m 32 ] ∗ [ 1 ] = m 32 (33)
For element l41:
l 41 = [ m 41 m 42 m 43 ] ∗ [ 1 l 21 l 31 ] (34)
l 41 = [ m 41 m 42 m 43 ] ∗ [ 1 m 21 m 31 + m 32 m 21 ] = m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 (35)
For element l42:
l 42 = [ m 42 m 43 ] ∗ [ 1 l 32 ] = [ m 42 m 43 ] ∗ [ 1 m 32 ] = m 42 + m 43 m 32 (36)
For element l43:
l 43 = [ m 43 ] (37)
This completes the L matrix for the 4 by 4 matrix shown in Equation (15), i.e.,
L = [ 1 0 0 0 m 21 1 0 0 m 31 + m 32 m 21 m 32 1 0 m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 m 42 + m 43 m 32 m 43 1 ] (15)
The number of operations required Np are related to the determination of the elements of the L matrix only. It is apparent that similar to the LU decomposition, the order of operations is of power 2, i.e., for n by n matrix the number of operations required grows proportional to n2. This is clearly seen as the number of lij across the rows for an arithmetic series, 1 , 2 , 3 , ⋯ , n − 1 which sums to:
N p = ( 1 + n − 1 2 ) ∗ ( n − 1 ) = n ( n − 1 ) 2 = O ( n 2 ) (38)
For example for a 4 by 4 L matrix
N p = 4 × ( 4 − 1 ) 2 = 6
The lij elements of the L matrix shown in Equation (5) show the six elements to be determined. Compared to the LU decomposition, the proposed method requires only half of the operations required for the LU decomposition. The reason is, unlike the LU method the L A X = L B = B ′ method does not require storage of the U elements, i.e., only the L matrix is needed to solve the system of linear equations.
The computation of the lij elements of the lower triangular matrix L can be carried out either row wise or column wise using more or less the same procedure as outlined in the following step by step procedure.
Step 1: Initially set all the Gaussian pivot row multipliers mrs of the element lij to zero values. During computation of a particular value of mrs the most recent values of the other pivot row multipliers will be used. In other words, the values of mrs will be updated once their values change because of successive row wise or column wise computation.
Step 2: Starting with the first column and second row and proceeding either row wise or column wise, calculate the mrs value for which r = i and s = j. For example for the element l21, the m value to be calculated is that of m21 and at l53 it would be m53 that will be calculated. The matrix equation for the computation of the m values is that of LA = U in which for the element lij the equation takes the form:
l i p a p j = u i j = 0 (39)
since uij is zero for the upper triangular matrix for i > j.
Step 3: Proceed likewise for all the elements mrs taking into account that fact that all the other m values are updated once a new value is computed for them as per step 2.
Step 4: After the computation of all the m values of the Gauss pivot multipliers is completed, form the L matrix elements lij using the summation rules of the permutation products involving the m products as given by Equation (24) and Equation (25) or using the matrix product given in Equation (30).
Step 5: Once the L matrix is formed compute the solution X vector of the system of equations AX = B using the formula shown in Equation (3), namely,
L A X = L B = B ′ (3)
In other words, the product LA results in the upper triangular matrix U which will allow the computation of the solution vector elements of X using back substitution.
As in the Gauss method, it is possible to check if a zero appears on the diagonal of the U = LA matrix, i.e., to check if uii = 0 for a given row i during the computation of the lij elements. In other words, for a given row i, a check can be made for the value of uii using the formula:
u i i = l i p a p i (40)
If the condition uii = 0 becomes true, row interchange can be made with rows from below in the equation.
Example 1:
The 4 × 4 system of linear equation shown below will be used to illustrate the
proposed method of solving systems of linear equations using direct decomposition of the A matrix, i.e. using the matrix reduction L A X = U X = L B = B ′ . The system of equation is:
[ 6 − 2 2 4 12 − 8 6 10 3 − 13 9 3 − 6 4 1 − 18 ] [ x 1 x 2 x 3 x 4 ] = [ 16 26 − 19 − 34 ]
Forming the lower triangular matrix L in the equation LA = U and using the undetermined Gauss pivot row multipliers mrs, the matrix equation LA = U becomes;
[ 1 0 0 0 m 21 1 0 0 m 31 + m 32 m 21 m 32 1 0 m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 m 42 + m 43 m 32 m 43 1 ] × [ 6 − 2 2 4 12 − 8 6 10 3 − 13 9 3 − 6 4 1 − 18 ] = [ u 11 u 12 u 13 u 14 0 u 22 u 23 u 24 0 0 u 33 u 34 0 0 0 u 44 ]
Method 1 (Column wise operation)
Column 1 operations:
Initially all the m values will be set to zero as outlined in the steps for solving the system of equations. Starting with column 1 and at row 2, the equation l 2 p a p 1 = u 21 = 0 gives;
m 21 ( 6 ) + 1 ( 12 ) = 0 ; m 21 = − 12 6 = − 2
Since row 2 operation is completed at this stage, check for the occurrence of a zero on the new pivot element, u 22 ≠ 0 ; u 22 = l 2 p a p 2 ≠ 0 ;
u 22 = m 21 ( − 2 ) + 1 ( − 8 ) = − 2 ( − 2 ) − 8 = − 4 ≠ 0 o.k.
For the third row operation at column 1;
l 3 p a p 1 = u 31 = 0 ; the unknown to be determined is m31
( m 31 + m 32 m 21 ) ( 6 ) + m 32 ( 12 ) + 1 ( 3 ) = u 31 = 0
Since m32 is as initially set zero and not yet determined, the above equation reduces to;
( m 31 + 0 ∗ m 21 ) ( 6 ) + 0 ( 12 ) + 1 ( 3 ) = u 31 = 0
m 31 = − 3 6 = − 1 2
For the fourth row at column 1;
l 4 p a p 1 = u 41 = 0 ; the unknown to be determined is m41
( m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ) ( 6 ) + m 42 + ( m 43 m 32 ) ( 12 ) + m 43 ( 3 ) + 1 ( − 6 ) = 0
Since all the m terms corresponding to columns 2 and 3 are as they were initially set zero (still undetermined) the above equation reduces to;
m 41 ( 6 ) + 1 ( − 6 ) = 0 ; m 41 = 6 6 = 1
Column 2 operations:
Starting with row 3;
u 32 = l 3 p a p 2 = 0 ;
( m 31 + m 32 m 21 ) ( − 2 ) + m 32 ( − 8 ) + 1 ( − 13 ) = 0
( − 1 2 + m 32 ( − 2 ) ) ( − 2 ) + m 32 ( − 8 ) + 1 ( − 13 ) = 0
1 + 4 m 32 − 8 m 32 − 13 = 0 ; m 32 = 12 − 4 = − 3
Since row 3 operation is completed at this stage, check the new pivot element u33, i.e.,
u 33 = l 3 p a p 3 ≠ 0 ;
( m 31 + m 32 m 21 ) ( 2 ) + m 32 ( 6 ) + 1 ( 9 ) = u 33 ≠ 0
( − 1 2 + ( − 3 ) ( − 2 ) ) ( 2 ) + ( − 3 ) ( 6 ) + 1 ( 9 ) = u 33 ≠ 0
( − 1 + 12 ) − 18 + 9 = 11 − 18 + 9 = 2 = u 33 ≠ 0 (acceptable)
Since u 33 ≠ 0 , there is no need for row interchange.
For row 4 column 2 operations;
u 42 = l 4 p a p 2 = 0 ;
( m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ) ( − 2 ) + ( m 42 + m 43 m 32 ) ( − 8 ) + m 43 ( − 13 ) + 1 ( 4 ) = 0
Since m43 = 0 (not yet determined), the above equation reduces to;
[ 1 + m 42 ( − 2 ) + ( 0 ) m 31 + ( 0 ) m 32 m 21 ] ( − 2 ) + ( m 42 + ( 0 ) m 32 ) ( − 8 ) + ( 0 ) ( − 13 ) + 1 ( 4 ) = 0
− 2 − 4 ∗ m 42 + 4 = 0
m 42 = − 2 − 4 = 1 2
Column 3 operations;
Proceeding to row 4 since the upper rows are already determined;
u 43 = l 4 p a p 3 = 0
[ m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ] ( 2 ) + [ m 42 + m 43 m 32 ] ( 6 ) + m 43 ( 9 ) + 1 ( 1 ) = 0
[ 1 + ( 1 2 ) ( − 2 ) + m 43 ( − 1 2 ) + m 43 ( − 3 ) ( − 2 ) ] ( 2 ) + [ 1 2 + m 43 ( − 3 ) ] ( 6 ) + m 43 ( 9 ) + 1 ( 1 ) = 0
2 ∗ ( − 1 2 + 6 ) m 43 + 3 − 18 ∗ m 43 + 9 m 43 + 1 = 0
11 ∗ m 43 − 18 ∗ m 43 + 9 ∗ m 43 + 4 = 0
2 ∗ m 43 + 4 = 0 ; m 43 = − 4 2 = − 2
Since row 4 is completed, check for the occurrence of a zero on the new pivot element, i.e., u44.
u 44 = l 4 p a p 4 ≠ 0 ;
[ m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ] ( 4 ) + [ m 42 + m 43 m 32 ] ( 10 ) + m 43 ( 3 ) + 1 ( − 18 ) ≠ 0
[ 1 + ( 1 2 ) ( − 2 ) + ( − 2 ) ( − 1 2 ) + ( − 2 ) ( − 3 ) ( − 2 ) ] ( 4 ) + [ 1 2 + ( − 2 ) ( − 3 ) ] ( 10 ) + ( − 2 ) ( 3 ) + 1 ( − 18 ) ≠ 0
[ 1 − 1 + 1 − 12 ] ( 4 ) + [ 1 2 + 6 ] ( 10 ) − 6 − 18 ≠ 0
− 44 + 65 − 6 − 18 = − 3 ≠ 0 (acceptable)
Now all the m pivot row multipliers are determined and the elements of the lower triangular matrix L can be determined as follows:
Column 1 elements
l 21 = m 21 = − 2
l 31 = m 31 + m 32 m 21 = ( − 1 2 ) + ( − 3 ) ( − 2 ) = − 1 2 + 6 = 11 2
l 41 = m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 = 1 + ( 1 2 ) ( − 2 ) + ( − 2 ) ( − 1 2 ) + ( − 2 ) ( − 3 ) ( − 2 )
l 41 = 1 − 1 + 1 − 12 = − 11
Column 2 elements:
l 32 = m 32 = − 3
l 42 = m 42 + m 43 m 32 = 1 2 + ( − 2 ) ( − 3 ) = 13 2
Column 3 elements:
l 43 = m 43 = − 2
Since all the l elements of the low triangular matrix are determined, the L matrix can now be written as follows:
L = [ 1 0 0 0 − 2 1 0 0 11 2 − 3 1 0 − 11 13 2 − 2 1 ]
Matrix Computation of the L Matrix for the Example 1The L matrix can be computed using the matrix form given by Equation 30, i.e.,
L L U ∗ L = [ 1 0 0 0 0 0 − m 21 1 0 0 0 0 − m 31 − m 32 1 0 0 0 − m 41 − m 42 − m 43 1 0 0 ⋮ ⋮ ⋮ ⋱ 1 0 − m n 1 − m n 2 − m n 3 ⋯ ⋯ 1 ] [ 1 0 0 0 0 0 l 21 1 0 0 0 0 l 31 l 32 1 0 0 0 l 41 l 42 l 43 1 0 0 ⋮ ⋮ ⋮ ⋱ 1 0 l n 1 l n 2 l n 3 ⋯ ⋯ 1 ] = I
For the above example, Equation (30) takes the form:
L L U ∗ L = [ 1 0 0 0 − m 21 1 0 0 − m 31 − m 32 1 0 − m 41 − m 42 − m 43 1 ] [ 1 0 0 0 l 21 1 0 0 l 31 l 32 1 0 l 41 l 42 l 43 1 ] = I
Substituting the computed m values
L L U ∗ L = [ 1 0 0 0 2 1 0 0 1 2 3 1 0 − 1 − 1 2 2 1 ] [ 1 0 0 0 l 21 1 0 0 l 31 l 32 1 0 l 41 l 42 l 43 1 ] = I
For element l21
2 + l 21 = 0 ; l 21 = − 2
For element l31:
1 2 + 3 ∗ l 21 + l 31 = 0
l 31 = − 1 2 − 3 ∗ ( − 2 ) = 11 2
For element l32:
3 + l 32 = 0
l 32 = − 3
For element l41:
− 1 ( 1 ) + ( − 1 2 ) ∗ l 21 + 2 ∗ l 31 + l 41 = 0
l 41 = 1 + ( 1 2 ) ∗ ( − 2 ) − 2 ∗ ( 11 2 ) = − 11
For element l42:
− 1 ∗ ( 1 2 ) + 2 ∗ l 32 + l 42 = 0
l 42 = 1 2 − 2 ∗ ( − 3 ) = 13 2
For element l43:
2 + l 43 = 0
l 43 = − 2
This completes the L matrix, i.e.,
L = [ 1 0 0 0 − 2 1 0 0 11 2 − 3 1 0 − 11 13 2 − 2 1 ]
The reduced matrix LA becomes;
L A = [ 1 0 0 0 − 2 1 0 0 11 2 − 3 1 0 − 11 13 2 − 2 1 ] [ 6 − 2 2 4 12 − 8 6 10 3 − 13 9 3 − 6 4 1 − 18 ] = [ 6 − 2 2 4 0 − 4 2 2 0 0 2 − 5 0 0 0 − 3 ]
Similarly, the operation L B = B ′ becomes;
B ′ = L B = [ 1 0 0 0 − 2 1 0 0 11 2 − 3 1 0 − 11 13 2 − 2 1 ] [ 16 26 − 19 − 34 ] = [ 16 − 6 − 9 − 3 ]
Finally the reduced equation L A X = L B = B ′ takes the form:
[ 6 − 2 2 4 0 − 4 2 2 0 0 2 − 5 0 0 0 − 3 ] [ x 1 x 2 x 3 x 4 ] = [ 16 − 6 − 9 − 3 ]
The elements of the solution vector X can now be determined by back substitution. Starting from the fourth row, x4 is determined;
x 4 = − 3 − 3 = 1
From the equation of row 3, x3 is determined;
2 ( x 3 ) − 5 ( 1 ) = − 9
x 3 = − 4 2 = − 2
Similarly using row 2 equation for x2;
− 4 ( x 2 ) + 2 ( − 2 ) + 2 ( 1 ) = − 6
x 2 = − 6 + 4 − 2 − 4 = − 4 − 4 = 1
Finally x1 is determined from equation of row 1;
6 ( x 1 ) − 2 ( 1 ) + 2 ( − 2 ) + 4 ( 1 ) = 16
x 1 = 16 + 2 + 4 − 4 6 = 18 6 = 3
Therefore the solution vector X is given by:
X T = { 3 1 − 2 1 }
This completes the solution using the proposed method. The alternative solution given below is only useful up to the computation of the Gaussian pivot row multiplier m following which the computation of the elements of the L and U matrix and the procedure for the determination of the solution vector X would be the same as was demonstrated above and need not be repeated.
Method 2 (Row wise operation)
Row 2 operations:
Initially all the m values will be set to zero as outlined in the steps for solving the system of equations. Starting with row 2 and at column 1, the equation l 2 p a p 1 = u 21 = 0 gives;
m 21 ( 6 ) + 1 ( 12 ) = 0 ; m 21 = − 12 6 = − 2
Since row 2 operation is completed at this stage, check for the occurrence of a zero on the new pivot element, u 22 ≠ 0 ;
u 22 = l 2 p a p 2 ≠ 0 ;
u 22 = m 21 ( − 2 ) + 1 ( − 8 ) = − 2 ( − 2 ) − 8 = − 4 ≠ 0 (acceptable)
Row 3 operations
For the third row operation at column 1;
l 3 p a p 1 = u 31 = 0 ; the unknown to be determined is m31
( m 31 + m 32 m 21 ) ( 6 ) + m 32 ( 12 ) + 1 ( 3 ) = u 31 = 0
Since the column 2 multiplier m32 is as initially set zero and not yet determined at this stage, the above equation reduces to;
( m 31 + 0 ∗ m 21 ) ( 6 ) + 0 ( 12 ) + 1 ( 3 ) = u 31 = 0
m 31 = − 3 6 = − 1 2
For row 3 column 2;
u 32 = l 3 p a p 2 = 0 ;
( m 31 + m 32 m 21 ) ( − 2 ) + m 32 ( − 8 ) + 1 ( − 13 ) = 0
( − 1 2 + m 32 ( − 2 ) ) ( − 2 ) + m 32 ( − 8 ) + 1 ( − 13 ) = 0
1 + 4 m 32 − 8 m 32 − 13 = 0 ; m 32 = 12 − 4 = − 3
Since row 3 operation is completed at this stage, check the new pivot element u33, i.e.,
u 33 = l 3 p a p 3 ≠ 0 ;
( m 31 + m 32 m 21 ) ( 2 ) + m 32 ( 6 ) + 1 ( 9 ) = u 33 ≠ 0
( − 1 2 + ( − 3 ) ( − 2 ) ) ( 2 ) + ( − 3 ) ( 6 ) + 1 ( 9 ) = u 33 ≠ 0
( − 1 + 12 ) − 18 + 9 = 11 − 18 + 9 = 2 = u 33 ≠ 0
Since u 33 ≠ 0 , there is no need for row interchange.
Row 4 operations
For the fourth row at column 1;
l 4 p a p 1 = u 41 = 0 ; the unknown to be determined is m41
( m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ) ( 6 ) + m 42 + ( m 43 m 32 ) ( 12 ) + m 43 ( 3 ) + 1 ( − 6 ) = 0
Since all the m terms corresponding to columns 2 and 3 belonging to row 4 are as they were initially set zero (still undetermined) the above equation reduces to;
m 41 ( 6 ) + 1 ( − 6 ) = 0 ; m 41 = 6 6 = 1
For row 4 column 2 operations;
u 42 = l 4 p a p 2 = 0 ;
( m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ) ( − 2 ) + ( m 42 + m 43 m 32 ) ( − 8 ) + m 43 ( − 13 ) + 1 ( 4 ) = 0
Since the row 4 column 3 multiplier, m43 = 0 (is not yet determined), the above equation reduces to;
[ 1 + m 42 ( − 2 ) + ( 0 ) m 31 + ( 0 ) m 32 m 21 ] ( − 2 ) + ( m 42 + ( 0 ) m 32 ) ( − 8 ) + ( 0 ) ( − 13 ) + 1 ( 4 ) = 0
− 2 − 4 ∗ m 42 + 4 = 0
m 42 = − 2 − 4 = 1 2
For row 4 column 3 operations:
u 43 = l 4 p a p 3 = 0
[ m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ] ( 2 ) + [ m 42 + m 43 m 32 ] ( 6 ) + m 43 ( 9 ) + 1 ( 1 ) = 0
[ 1 + ( 1 2 ) ( − 2 ) + m 43 ( − 1 2 ) + m 43 ( − 3 ) ( − 2 ) ] ( 2 ) + [ 1 2 + m 43 ( − 3 ) ] ( 6 ) + m 43 ( 9 ) + 1 ( 1 ) = 0
2 ∗ ( − 1 2 + 6 ) m 43 + 3 − 18 ∗ m 43 + 9 m 43 + 1 = 0
11 ∗ m 43 − 18 ∗ m 43 + 9 ∗ m 43 + 4 = 0
2 ∗ m 43 + 4 = 0 ; m 43 = − 4 2 = − 2
Since row 4 is completed, check for the occurrence of a zero on the new pivot element, i.e., u44.
u 44 = l 4 p a p 4 ≠ 0 ;
[ m 41 + m 42 m 21 + m 43 m 31 + m 43 m 32 m 21 ] ( 4 ) + [ m 42 + m 43 m 32 ] ( 10 ) + m 43 ( 3 ) + 1 ( − 18 ) ≠ 0
[ 1 + ( 1 2 ) ( − 2 ) + ( − 2 ) ( − 1 2 ) + ( − 2 ) ( − 3 ) ( − 2 ) ] ( 4 ) + [ 1 2 + ( − 2 ) ( − 3 ) ] ( 10 ) + ( − 2 ) ( 3 ) + 1 ( − 18 ) ≠ 0
[ 1 − 1 + 1 − 12 ] ( 4 ) + [ 1 2 + 6 ] ( 10 ) − 6 − 18 ≠ 0
− 44 + 65 − 6 − 18 = − 3 ≠ 0 (acceptable)
Now all the m pivot row multipliers are determined and the determination of the L and U matrices as well as the computation of the solution vector X would proceed in exactly the same manner as demonstrated in method 1, column wise operations.
The example provided above shows in clear steps the solution step for solving system of linear equations using direct decomposition of the form L A X = L B = B ′ . Once the L matrix is formed, it can be used to solve any variants of the equation AX = B in which the right hand side column vector B is changed. This is demonstrated in the example given below.
Example 2: let the right hand side column vector be changed to the following while the coefficient matrix A remains the same. The new column vector B is given as:
B T = { 4 6 26 69 }
In this case, since the L matrix in the equation LA has already been worked out, the only additional operation needed would be the computation of L B = B ′ . Equation (3), namely,
L A X = L B = B ′
would be used to determine the new solution vector X. Starting with the computation of L B = B ′ ;
L B = B ′ = [ 1 0 0 0 − 2 1 0 0 11 2 − 3 1 0 − 11 13 2 − 2 1 ] [ 4 6 26 69 ] = [ 4 − 2 30 12 ]
Finally, the solution vector X is computed from L A X = L B = B ′ :
[ 6 − 2 2 4 0 − 4 2 2 0 0 2 − 5 0 0 0 − 3 ] [ x 1 x 2 x 3 x 4 ] = [ 4 − 2 30 12 ]
The elements of the solution vector X can now be determined by back substitution. Starting from the fourth row, x4 is determined;
x 4 = 12 − 3 = − 4
From the equation of row 3, x3 is determined;
2 ( x 3 ) − 5 ( − 4 ) = 30
x 3 = 30 − 20 2 = 5
Similarly using row 2 equation for x2;
− 4 ( x 2 ) + 2 ( 5 ) + 2 ( − 4 ) = − 2
x 2 = − 2 + 8 − 10 − 4 = − 4 − 4 = 1
Finally x1 is determined from equation of row 1;
6 ( x 1 ) − 2 ( 1 ) + 2 ( 5 ) + 4 ( − 4 ) = 4
x 1 = 4 + 16 − 10 + 2 6 = 12 6 = 2
Therefore, the solution vector X is given by:
X T = { 2 1 5 − 4 }
The proposed method, developed and demonstrated with examples so far, shows that solution to linear systems of equation can be obtained through direct decomposition of the A matrix using the operation L A X = L B = B ′ . The method provides a clear procedure for direct computation of the L matrix, the only matrix that is needed to transform the original equation AX = B in to a reduced form, i.e., LAX = BX unlike for example the LU method which requires that both the L and U matrix be stored to find the solution through A X = L U X = B . The elements lij of the lower triangular matrix L are shown to be sums of permutation products of the Gauss pivot row multipliers mrs. The relationship between lij and mrs is clearly established through a formula and it is easy to visually construct this relationship using a tree diagram that will assist in easy memorisation of the relationship. In addition (and as an alternative procedure) the relationship so established between elements lij of the lower triangular matrix L and the Gauss pivot row multipliers mrs enables construction of the L matrix directly from the Gauss elimination steps.
The characteristic of Gauss elimination method is that the reduction to an upper triangular matrix can only proceed column wise. It is not possible to proceed row wise in the Gauss method. On the other hand, the LU decomposition requires alternate transition between the L and U elements for determining the LU compact matrix. By contrast, the proposed LA = U reduction method can proceed either column wise or row wise essentially giving the same result. This flexibility is demonstrated in the example shown above where it is easily seen that the computation of the Gauss pivot row multipliers remains more or less the same for both the row wise and column wise operations.
The storage requirement during the reduction process is related to the generation of the L matrix. Unlike the LU method, storage is needed only for the L matrix since the solution directly proceeds from the reduction L A X = L B = B ′ in which there is no need to store the U matrix. The number of elements that need change is of the order O(n2) as shown in Equation 38 and is typically half the number of operations required for the LU decomposition because in the LU decomposition both the L and U elements need to be determined and stored.
A direct decomposition of the coefficient matrix forming part of a system of linear equations using a single lower triangular reducing matrix L has been demonstrated as shown in this paper. The method allows solution to the system of linear equations to proceed through storage of a single lower triangular matrix L only, through which both the coefficient matrix A and the right hand side column vector B are transformed. Elements of the reducing matrix L are shown to be sums of permutation products of the pivot row multipliers of the Gauss elimination technique. These sums of permutation products, for any element of the reducing matrix L, can be easily constructed using a tree diagram that is relatively easy to memorize besides using the formula developed for the purpose. These L matrix elements can also be alternatively computed using matrix products. In the process of determining the elements of the L matrix, either row wise or column wise procedure can be followed essentially giving the same result which provides added flexibility to the proposed method. Equivalence of this newly proposed method with both the Gauss elimination and LU decomposition techniques has been established. In the case of the equivalence with Gauss elimination technique, elements of the L matrix are specified as functions of the Gauss pivot row multipliers. This also implies that it is possible to construct the reducing L matrix of the proposed direct decomposition method using the Gauss pivot row multipliers. As has been demonstrated, the L matrix can be directed constructed from the Gauss pivot row multipliers using the matrix product LLuL = I. For the LU decomposition, the L matrix of the proposed method is simply the inverse of the L matrix of the LU decomposition. In terms of storage of computed values, it can be seen that the proposed method of direct decomposition using the transformation L A X = L B = B ′ needs only storage of the L matrix elements which is half in size compared with storage of all the L and U elements in the LU decomposition method.
Apart from providing added flexibility and simplicity, the proposed method would be of good educational value providing an alternative procedure for solving systems of linear equations.
The authors declare no conflicts of interest regarding the publication of this paper.
Tiruneh, A.T., Debessai, T.Y., Bwembya, G.C. and Nkambule, S.J. (2019) The LA = U Decomposition Method for Solving Systems of Linear Equations. Journal of Applied Mathematics and Physics, 7, 2031-2051. https://doi.org/10.4236/jamp.2019.79140