Solving Linear Equation System

1. Sulaiman M. KARIM
man_musaria@yahoo.com
Solving Linear Equation System

2. Outline
 Introduction
 Doolittle-crout method
 Cholesky method

3. Introduction
A linear equation in one unknown is an equation of the form
a x = b, where a and b are constants and x is an unknown that
we wish to solve for. Similarly, a linear equation in n unknowns
x1, x2, …, xn is an equation of the form: a1 · x1 + a2 · x2 + … +
an · xn = b, where a1, a2, …, an and b are constants. The name
linear comes from the fact that such an equation in two
unknowns or variables represents a straight line. A set of such
equations is called a system of linear equations.
In mathematics, a system of linear equations is a collection
of two or more linear equations with the same set of
variables in all the equations.

4. Some types of linear equation system:
 Elimination method
 Gauss elimination
 Gauss elimination with partial pivoting
 Iteration method
 Jacobi
 Gauss-seidel
 LU factorization/decomposition method
 Doolittle
 Crout
 Cholesky
 Thomas method
 Thomas algorithm

5. LU factorization/decomposition method
For a linear system
Ax = b
Use substitution of A = LU, where L is a
lower triangle
matrix, and U is upper triangular matrix.
LUx = b
Let
Ux = Y
Yields
LY = b

6. *There are a different between Doolittle, crout and Cholesky
method:
If [L] has ones on its diagonal, then you've done a Doolittle factorization.
If [U] has ones on its diagonal, then you've done a Crout factorization.
If [L] is the transpose of U, then you've done a Cholesky factorization.
[A] [L] [U]
[A] [L] [U]

7. Example one-Doolittle method
Doolittle’s Method LU factorization of A when the diagonal elements of lower
triangular matrix, L have a unit value.
x₁ + x₂ + x₃ =5
x ₁ + 2x₂ +2x₃ =6
x ₁ + 2x₂ +3x₃ =8
Step 1: Create matrices A, X and B , where A is the augmented matrix, X constitutes
the variable vectors and B are the constants
1 1
1
1 2
2
1 2
3
A =
X ₁
X ₂
X ₃
X =
5
6
8
B=

8. Step2: Let A = LU, where L is the lower triangular matrix and U is the upper
triangular matrix assume that the diagonal entries L is equal to 1
Let A=LU
1 1
1
1 2
2
1 2
3
=
1 0
0
a 1
0
b c
1
d e
f
0 g
h
0 0
i
d e f
ad ae + g af
+ h
bd be + cg bf +
ch + i
=
*Find the values e=1d=
1
f=1
ad=1 Ae+g=2 Af+h=2
a=1 g=1 h=1bd=1
be+cg=2 Bf+ch+i=3 i=1b=1
c=1
1 1
1
1 2
2
1 2
3

9. Step3: Let Ly = B, solve for y’s
Y₁ =5
Y₁ +Y₂ =6
Y₁ + Y₂ + Y₃ =8
Y₁ =5, Y₂ =1, Y₃ =2
Let Ly=B
1 0
0
1 1
0
1 1
1
Y ₁
Y ₂
Y ₃
5
6
8
=

10. Step3: Let Ux = y, solve for the variable vectors x
X₃ =2
X₂ +X₃ =1
X₁ +X₂ +X₃ =5
X₁ =4, X₂ =-1, X₃ =2
X=
Let Ux=y
X ₁
X ₂
X ₃
5
1
2
=
4
-
1
2
1 1
1
0 1
1
0 0
1

11. Example two-Doolittle method
2x - y - 2z = -1
-4x + 6y +3z =13
-4x - 2y +8z = -6
Step 1: Create matrices A, X and B , where A is the augmented matrix, X constitutes
the variable vectors and B are the constants
2 -1
-2
-4 6
3
-4 -2
8
A =
X
Y
Z
X =
-1
1
3
-6
B=

12. Step2: Let A = LU, where L is the lower triangular matrix and U is the upper
triangular matrix assume that the diagonal entries L is equal to 1
Let A=LU =
1 0
0
a 1
0
b c
1
d e
f
0 g
h
0 0
i
d e f
ad ae + g af
+ h
bd be + cg bf +
ch + i
=
*Find the values e= -1d=
2
f=-2
ad= -4 Ae+g=6 Af+h=3
a= -2 g=4 h= -1bd= -4
be+cg= -2 Bf+ch+i=8 i=3b= -2
c= -1
2 -1
-2
-4 6
3
-4 -2
8
2 -1
-2
-4 6
3
-4 -2
8

13. Step3: Let Ly = B, solve for y’s
Y₁ = -1, Y₂ =11, Y₃ =3
Let Ly=B
1 0
0
-2 1
0
-2 -1
1
Y ₁
Y ₂
Y ₃
-1
1
3
-6
=

14. Step4: Let Ux = y, solve for the variable vectors x
X₁ =2, X₂ =3, X₃ =1
X=
Let Ux=y
2 -1
2
0 4
-1
0 0
3
X ₁
X ₂
X ₃
-1
1
1
3
=
2
3
1

15. Example three-Crout method
5x₁ + 4x₂ + x₃ =3.4
10x ₁ + 9x₂ +4x₃ =8.8
10x ₁ + 13x₂ +15x₃ =19.2
Step 1: Create matrices A, X and B , where A is the augmented matrix, X constitutes
the variable vectors and B are the constants
5 4
1
10 9
4
10 13
15
A =
X ₁
X ₂
X ₃
X =
3.4
8.8
19.
2
B=

16. Step2: Let A = LU, where L is the lower triangular matrix and U is the upper
triangular matrix assume that the diagonal entries U is equal to 1
Let A=LU =
a 0
0
b c
0
d e
f
1 g
h
0 1
i
0 0
1
a ag
ah
b bg + c
bh + ci
d dg + e dh +
ei + f
=
*Find the values b=
10
a=5 f=2
g=4/5 c=1 e=5
i=2 f=3h=1/5
5 4
1
10 9
4
10 13
15
5 4
1
10 9
4
10 13
15

17. Step3: Let Ly = B, solve for y’s
5y₁ =3.4, y₁ =0.68
10y₁ +y₂ =8.8, y₂ =2
10y₁ +5y₂ +3y₃ =19.2, y₃ =0.80
Let Ly=B
Y ₁
Y ₂
Y ₃
=
5 0
0
10 1
0
10 5
3
3.4
8.8
19.
2

18. Step4: Let Ux = y, solve for the variable vectors x
X3= 0.80
X2+2x3=2
x1+ 4/5x2 +1/5x3=0.68
X=
Let Ux=y
X ₁
X ₂
X ₃
=
0.2
0
0.4
0
0.8
0
1 4/5
1/5
0 1
2
0 0
1
0.6
8
2
0.8
0

19. Example four-Crout method
x + 2y + z= 4
2x + 3y - z = -3
3x + y +2z = 3
Step 1: Create matrices A, X and B , where A is the augmented matrix, X constitutes
the variable vectors and B are the constants
1 2
1
2 -3
-1
3 1
2
A =
X
Y
Z
X =
4
-3
3
B=

20. Step2: Let A = LU, where L is the lower triangular matrix and U is the upper
triangular matrix assume that the diagonal entries U is equal to 1
Let A=LU =
a 0
0
b c
0
d e
f
1 g
h
0 1
i
0 0
1
a ag
ah
b bg + c
bh + ci
d dg + e dh +
ei + f
=
*Find the values b= 2a=1 f=8/7
g=2 c= -7 e= -5
i=3/7 f=3h=1
1 2
1
2 -3 -
1
3 1
2
1 2
1
2 -3 -
1
3 1
2

21. Step3: Let Ly = B, solve for y’s
y₁ = 4, y₁ =4
2y₁ - 7y₂ = -3, y₂ =11/7
3y₁ - 5y₂ + 8/7y₃ =3, y₃ = -1
Let Ly=B
Y ₁
Y ₂
Y ₃
=
1 0
0
2 -7
0
3 -5
8/7
4
-3
3

22. Step4: Let Ux = y, solve for the variable vectors x
X+2y+z=4
y+3/7z=11/7
Z=-1
Let Ux=y
X
Y
Z
=
x=1
Y=2
Z=-1
1 2
1
0 1
3/7
0 0
1
4
11/7
-1

23. Example five-Cholesky’s Method:
The Cholesky’s method, unlike the Doolittle’s and Crout’s does not have any
condition for the main diagonal entries. The matrix should be symmetric and for a
symmetric, positive definitive matrix.
4x₁ + 10x₂ + 8x₃ = 44
10x₁ + 26x₂ + 26x₂ = 128
8x₁ + 26x₂ + 61x₂ = 214
Step1: create matrix A, X and B.4 10
8
10 26
26
8 26
61
X ₁
X ₂
X ₃
44
12
8
21
4
A= B=X=

24. Step2: Let A = LLᵗ
A=LLᵗ =
a 0
0
b c
0
d e
f
a b
c
0 d
e
0 0
f
a² ag
ad
ab b² + c² bd +
ce
ad dg + e d² + e²
+ f²
=
*Find the values b= 5a=2 d=4
c=1 e=6f=3
4 10
8
10 26
26
8 26
61
4 10
8
10 26
26
8 26
61

25. Step3: Let Ly = B
2 0
0
5 1
0
4 6
3
Y ₁
Y ₂
Y ₃
44
12
8
21
4
=
Y ₁ = 22
Y ₂ = 18
Y ₃ = 6
Ly = B

26. Step4: Lᵗ x = y, then solve for x
x=2
x₂=6
2x₁ + 5x₂ + 4x ₃ = 22; x₁= -8
22
18
6
=Lᵗ x = y
X ₁
X ₂
X ₃
2 5
4
0 1
6
0 0
3

27. Example Six-Cholesky’s Method :
x₁ - x₂ + 2x₃ = 8
-x₁ + 5x₂ - 4x₃ = 2
2x₁ + 4x₂ + 6x₃ = 6
Step1: create matrix A, X and B.
1 -1
2
-1 5
-4
2 -4
6
X ₁
X ₂
X ₃
8
2
6
A= B=X=

28. Step2: Let A = LLᵗ
A=LLᵗ =
a 0
0
b c
0
d e
f
a b
c
0 d
e
0 0
f
a² ag
ad
ab b² + c² bd +
ce
ad dg + e d² + e²
+ f²
=
*Find the values b= -
1
a=1 d=2
c=2 e=-1f= 1
1 -1
2
-1 5
-4
2 -4
6
1 -1
2
-1 5
-4
2 -4
6
bd= -2

29. Step3: Let Ly = B
1 0
0
-1 2
0
2 -1
1
Y ₁
Y ₂
Y ₃
8
2
6
=
Y ₁ = 8
Y ₂ = 5
Y ₃ = -5
Ly = B

30. Step4: Lᵗ x = y, then solve for x
x1=18
X2=0
X3=-5
8
5
-5
=Lᵗ x = y
X ₁
X ₂
X ₃
1 -1
2
0 2
-1
0 0
1

31. Complexity of LU Decomposition
to solve Ax=b:
– decompose A into LU --
cost 2n³/3 flops
– solve Ly=b for y by forw. substitution
– solve Ux=y for x by back substitution
slower alternative:
– compute Aˉ¹ -- cost 2n³ flops --
cost n² flops
– multiply x=A ˉ¹ b -- cost 2n² flops --
cost n² flops
this costs about 3 times as much as LU