The document discusses topics related to matrices and systems of linear equations. It begins by providing examples of different types of matrices, such as identity, zero, diagonal, triangular, and symmetric matrices. It then discusses operations that can be performed on matrices, such as addition, multiplication and determining the determinant. The document also covers solving systems of linear equations using techniques like Gaussian elimination, Cramer's rule and graphical methods. It notes that Cramer's rule is not practical for large systems and discusses issues like singular, inconsistent and ill-conditioned systems of equations.

1. Topic3 Saed Sasi 1
Solution of Systems of Linear Equations
Read Chapter 9 and 11 of the textbook

2. Topic3 Saed Sasi 2
Vector, Matrices, and
Linear Equations

3. Topic3 Saed Sasi 3
MATRICES




































1
2
0
0
1
4
1
0
0
1
4
3
0
0
2
1
l
Tridiagona
,
6
0
0
0
0
0
0
0
0
0
4
0
0
0
0
1
diagonal
1
0
0
1
matrix
identity
0
0
0
0
0
0
matrix
zero
:
Examples
numbers
of
array
l
dimensiona
two
a
:
Matrix

4. Topic3 Saed Sasi 4
MATRICES
























1
0
0
0
1
4
0
0
0
1
4
0
3
1
2
1
ngular
upper tria
,
4
5
1
5
0
1
1
1
2
symmetric
:
Examples

5. Topic3 Saed Sasi 5
Determinant of a MATRICES
82
)
0
15
(
1
)
5
12
(
1
)
25
(
2
5
0
1
-
3
1
-
4
5
1
-
3
1
-
4
5
5
0
2
4
5
1
5
0
1
1
3
2
det
:
Examples
only
matrices
square
for
Defined






















6. Topic3 Saed Sasi 6
Adding and Multiplying Matrices
j
i
j
i
m
k
,
b
a
c
B
A
C
*
p
m
if
only
defined
is
AB
C
product
The
*
q)
B(p
and
m)
(n
A
matrices
two
of
tion
Multiplica
,
b
a
c
B
A
C
*
size
same
the
have
they
if
only
Defined
*
B
and
A
matrices
two
of
addition
The
1
kj
ik
ij
ij
ij
ij

















7. Topic3 Saed Sasi 7
Systems of Linear Equations
form
Matrix
form
Standard
7
5
3
6
0
1
3
1
5
.
2
3
4
2
7
6
5
3
5
.
2
3
3
4
2
forms
different
in
presented
be
can
equations
linear
of
system
A
3
2
1
3
1
3
2
1
3
2
1
















































x
x
x
x
x
x
x
x
x
x
x

8. Topic3 Saed Sasi 8
Solutions of Linear Equations
5
2
3
:
equations
following
the
o
solution t
a
is
2
1
2
1
2
1
2
1

















x
x
x
x
x
x

9. Topic3 Saed Sasi 9
Solutions of Linear Equations
 Some systems of equations may have infinite
number of solutions
all
for
solution
a
is
)
3
(
5
.
0
solutions
of
number
infinite
have
6
4
2
3
2
2
1
2
1
2
1
a
a
a
x
x
x
x
x
x



















10. Topic3 Saed Sasi 10
Solutions of Linear Equations
 A set of equations is inconsistent if there
exists no solution to the system of equations:
nt
inconsiste
are
equations
These
5
4
2
3
2
2
1
2
1




x
x
x
x
Both types of systems are said to be singular.

11. Topic3 Saed Sasi 11
In addition, systems that are very close to being singular
(Fig. 9.2c) can also cause problems. These systems are said
to be ill-conditioned. Graphically, this corresponds to the fact
that it is difficult to identify the exact point at which the lines
intersect. Ill-conditioned systems will also pose problems
when they are encountered during the numerical solution of
linear equations. This is because they will be extremely
sensitive to round-off error

12. Topic3 Saed Sasi 12
Graphical Solution of Systems of
Linear Equations
5
2
3
2
1
2
1




x
x
x
x
solution
x1
x2

13. Topic3 Saed Sasi 13
Cramer’s Rule is Not Practical
way
efficient
in
computed
are
ts
determinan
the
if
used
be
can
It
system.
30
by
30
a
solve
to
years
10
needs
computer
super
A
.
systems
large
for
practical
not
is
Rule
s
Cramer'
2
2
1
1
1
5
1
3
1
,
1
2
1
1
1
2
5
1
3
system
the
solve
to
used
be
can
Rule
s
Cramer'
17
2
1 


 x
x

14. Topic3 Saed Sasi 14
 Naive Gaussian Elimination
 Examples
Lecture 13
Naive Gaussian Elimination

15. Topic3 Saed Sasi 15
Naive Gaussian Elimination
o The method consists of two steps
o Forward Elimination: the system is reduced to
upper triangular form. A sequence of elementary
operations is used.
o Backward Substitution: Solve the system starting
from the last variable. Solve for xn ,xn-1,…x1.































































'
'
'
0
0
'
'
0
3
2
1
3
2
1
33
23
22
13
12
11
3
2
1
3
2
1
33
32
31
23
22
21
13
12
11
b
b
b
x
x
x
a
a
a
a
a
a
b
b
b
x
x
x
a
a
a
a
a
a
a
a
a

16. Topic3 Saed Sasi 16
Example 1
17
7
5
6
4
8
3
2
1
1
3
3
3
7
2
3
1
1
2
2
2
10
2
3
2
)
(
1
8
3
2
3
,
2
equations
from
Eliminate
:
Step1
___
n
Eliminatio
Forward
:
1
Part
:
n
Eliminatio
Gaussian
Naive
using
Solve
3
2
3
2
3
2
1
3
2
1
3
2
1
3
2
1
1




































x
x
x
x
x
x
x
eq
eq
eq
x
x
x
eq
eq
eq
x
x
x
equation
pivot
unchanged
eq
x
x
x
x

17. Topic3 Saed Sasi 17
Example 1



































13
13
6
4
8
3
2
2
1
5
3
3
17
7
5
)
(
2
6
4
1
8
3
2
3
equation
from
Eliminate
:
Step2
n
Eliminatio
Forward
:
1
Part
3
3
2
3
2
1
3
2
3
2
3
2
1
2
x
x
x
x
x
x
eq
eq
eq
x
x
equation
pivot
unchanged
eq
x
x
unchanged
eq
x
x
x
x

18. Topic3 Saed Sasi 18
Example 1
Backward Substitution






































1
2
1
is
solution
The
1
3
2
8
2
1
4
6
1
13
13
3
2
1
1
,
1
3
2
1
,
1
3
3
,
1
2
2
,
1
1
1
3
2
,
2
3
3
,
2
2
2
3
,
3
3
3
x
x
x
a
x
x
a
x
a
x
a
b
x
x
a
x
a
b
x
a
b
x

19. Forward Elimination
A set of n equations and n unknowns
1
1
3
13
2
12
1
11 ... b
x
a
x
a
x
a
x
a n
n 




2
2
3
23
2
22
1
21 ... b
x
a
x
a
x
a
x
a n
n 




n
n
nn
n
n
n b
x
a
x
a
x
a
x
a 



 ...
3
3
2
2
1
1
. .
. .
. .
(n-1) steps of forward elimination

20. Forward Elimination
Step 1
For Equation 2, divide Equation 1 by and
multiply by .
)
...
( 1
1
3
13
2
12
1
11
11
21
b
x
a
x
a
x
a
x
a
a
a
n
n 










1
11
21
1
11
21
2
12
11
21
1
21 ... b
a
a
x
a
a
a
x
a
a
a
x
a n
n 



11
a
21
a

21. Forward Elimination
1
11
21
1
11
21
2
12
11
21
1
21 ... b
a
a
x
a
a
a
x
a
a
a
x
a n
n 



1
11
21
2
1
11
21
2
2
12
11
21
22 ... b
a
a
b
x
a
a
a
a
x
a
a
a
a n
n
n 





















'
2
'
2
2
'
22 ... b
x
a
x
a n
n 


2
2
3
23
2
22
1
21 ... b
x
a
x
a
x
a
x
a n
n 




Subtract the result from Equation 2.
−
_________________________________________________
or

22. Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations as
1
1
3
13
2
12
1
11 ... b
x
a
x
a
x
a
x
a n
n 




'
2
'
2
3
'
23
2
'
22 ... b
x
a
x
a
x
a n
n 



'
3
'
3
3
'
33
2
'
32 ... b
x
a
x
a
x
a n
n 



'
'
3
'
3
2
'
2 ... n
n
nn
n
n b
x
a
x
a
x
a 



. . .
. . .
. . .
End of Step 1

23. Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
Forward Elimination
1
1
3
13
2
12
1
11 ... b
x
a
x
a
x
a
x
a n
n 




'
2
'
2
3
'
23
2
'
22 ... b
x
a
x
a
x
a n
n 



"
3
"
3
3
"
33 ... b
x
a
x
a n
n 


"
"
3
"
3 ... n
n
nn
n b
x
a
x
a 


. .
. .
. .
End of Step 2

24. Forward Elimination
At the end of (n-1) Forward Elimination steps, the
system of equations will look like
'
2
'
2
3
'
23
2
'
22 ... b
x
a
x
a
x
a n
n 



"
3
"
3
3
"
33 ... b
x
a
x
a n
n 


   
1
1 


n
n
n
n
nn b
x
a
. .
. .
. .
1
1
3
13
2
12
1
11 ... b
x
a
x
a
x
a
x
a n
n 




End of Step (n-1)

25. Matrix Form at End of Forward
Elimination

















































 )
(n-
n
"
'
n
)
(n
nn
"
n
"
'
n
'
'
n
b
b
b
b
x
x
x
x
a
a
a
a
a
a
a
a
a
a
1
3
2
1
3
2
1
1
3
33
2
23
22
1
13
12
11
0
0
0
0
0
0
0











26. Back Substitution Starting Eqns
'
2
'
2
3
'
23
2
'
22 ... b
x
a
x
a
x
a n
n 



"
3
"
3
"
33 ... b
x
a
x
a n
n 


   
1
1 


n
n
n
n
nn b
x
a
. .
. .
. .
1
1
3
13
2
12
1
11 ... b
x
a
x
a
x
a
x
a n
n 





27. Back Substitution
Start with the last equation because it has only one unknown
)
1
(
)
1
(


 n
nn
n
n
n
a
b
x

28.    
  1
,...,
1
for
1
1
1
1




 




n
i
a
x
a
b
x i
ii
n
i
j
j
i
ij
i
i
i
)
1
(
)
1
(


 n
nn
n
n
n
a
b
x

29. Topic3 Saed Sasi 29
How Do We Know If a Solution is
Good or Not
 Given AX=B
 X is a solution if AX-B=0
 Due to computation error AX-B may not be zero
 Compute the residuals R=|AX-B|
 One possible test is ?????


i
i
r
max
if
acceptable
is
solution
The

30. Topic3 Saed Sasi 30
Determinant
13
det
det
13
0
0
4
1
0
3
2
1
A'
2
1
3
2
3
2
3
2
1
A
:
Example
t
determinan
affect the
not
do
operations
elementary
The
operations
Elementary





















 












(A')
(A)

31. Naïve Gauss Elimination
Pitfalls

32. 9
5
5
11
3
2
6
3
7
10
3
2
1
3
2
1
3
2








x
x
x
x
x
x
x
x
Pitfall#1. Division by zero

































9
11
3
5
1
5
3
2
6
7
10
0
3
2
1
x
x
x

33. Is division by zero an issue here?
9
5
5
14
3
5
6
15
7
10
12
3
2
1
3
2
1
3
2
1









x
x
x
x
x
x
x
x
x

































9
14
15
5
1
5
3
5
6
7
10
12
3
2
1
x
x
x

34. Is division by zero an issue here?
YES
28
5
24
14
3
5
6
15
7
10
12
3
2
1
3
2
1
3
2
1









x
x
x
x
x
x
x
x
x

































28
14
15
5
1
24
3
5
6
7
10
12
3
2
1
x
x
x


































2
5
.
6
15
19
21
12
5
.
6
0
0
7
10
12
3
2
1
x
x
x
Division by zero is a possibility at any step
of forward elimination

35. Pitfall#2. Large Round-off Errors

































9
751
.
1
45
3
1
5
7
249
.
2
3
10
15
20
3
2
1
x
x
x
Exact Solution





















1
1
1
3
2
1
x
x
x

36. Pitfall#2. Large Round-off Errors

































9
751
.
1
45
3
1
5
7
249
.
2
3
10
15
20
3
2
1
x
x
x
Solve it on a computer using 6 significant digits with chopping





















999995
.
0
05
.
1
9625
.
0
3
2
1
x
x
x

37. Pitfall#2. Large Round-off Errors

































9
751
.
1
45
3
1
5
7
249
.
2
3
10
15
20
3
2
1
x
x
x
Solve it on a computer using 5 significant digits with chopping





















99995
.
0
5
.
1
625
.
0
3
2
1
x
x
x
Is there a way to reduce the round off error?

38. Avoiding Pitfalls
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero

39. Avoiding Pitfalls
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error

40. Gauss Elimination with
Partial Pivoting

41. Pitfalls of Naïve Gauss Elimination
• Possible division by zero
• Large round-off errors

42. Avoiding Pitfalls
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero

43. Avoiding Pitfalls
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error

44. What is Different About Partial
Pivoting?
pk
a
At the beginning of the kth step of forward elimination,
find the maximum of
nk
k
k
kk a
a
a .......,
,.........
, ,
1

If the maximum of the values is
in the p th row, ,
n
p
k 
 then switch rows p and k.

45. Example































2
279
2
177
8
106
1
12
144
1
8
64
1
5
25
3
2
1
.
.
.
a
a
a
Solve the following set of equations
by Gaussian elimination with partial
pivoting

46. Example 2 Cont.
































2
279
2
177
8
106
1
12
144
1
8
64
1
5
25
3
2
1
.
.
.
a
a
a
1. Forward Elimination
2. Back Substitution










2
.
279
1
12
144
2
.
177
1
8
64
8
.
106
1
5
25




47. Forward Elimination

48. Number of Steps of Forward
Elimination
Number of steps of forward elimination is
(n1)=(31)=2

49. Forward Elimination: Step 1
• Examine absolute values of first column, first row
and below.
144
,
64
,
25
• Largest absolute value is 144 and exists in row 3.
• Switch row 1 and row 3.





















8
.
106
1
5
25
2
.
177
1
8
64
2
.
279
1
12
144
2
.
279
1
12
144
2
.
177
1
8
64
8
.
106
1
5
25







50. Forward Elimination: Step 1 (cont.)
.
   
1
.
124
4444
.
0
333
.
5
99
.
63
4444
.
0
2
.
279
1
12
144 
 











8
.
106
1
5
25
2
.
177
1
8
64
2
.
279
1
12
144



 
 
 
10
.
53
.5556
0
667
.
2
0
124.1
0.4444
5.333
63.99
177.2
1
8
64














8
.
106
1
5
25
10
.
53
5556
.
0
667
.
2
0
2
.
279
1
12
144



Divide Equation 1 by 144 and
multiply it by 64, .
4444
.
0
144
64

Subtract the result from
Equation 2
Substitute new equation for
Equation 2

51. Forward Elimination: Step 1 (cont.)
.
   
47
.
48
1736
.
0
083
.
2
00
.
25
1736
.
0
279.2
1
12
144 
 

 
 
 
33
.
58
8264
.
0
917
.
2
0
48.47
0.1736
2.083
25
106.8
1
5
25














8
.
106
1
5
25
10
.
53
5556
.
0
667
.
2
0
2
.
279
1
12
144













33
.
58
8264
.
0
917
.
2
0
10
.
53
5556
.
0
667
.
2
0
2
.
279
1
12
144



Divide Equation 1 by 144 and
multiply it by 25, .
1736
.
0
144
25

Subtract the result from
Equation 3
Substitute new equation for
Equation 3

52. Forward Elimination: Step 2
• Examine absolute values of second column, second row
and below.
2.917
,
667
.
2
• Largest absolute value is 2.917 and exists in row 3.
• Switch row 2 and row 3.





















10
.
53
5556
.
0
667
.
2
0
33
.
58
8264
.
0
917
.
2
0
2
.
279
1
12
144
33
.
58
8264
.
0
917
.
2
0
10
.
53
5556
.
0
667
.
2
0
2
.
279
1
12
144







53. Forward Elimination: Step 2 (cont.)
.
   
33
.
53
7556
.
0
667
.
2
0
9143
.
0
58.33
0.8264
2.917
0 
 











10
.
53
5556
.
0
667
.
2
0
33
.
58
8264
.
0
917
.
2
0
2
.
279
1
12
144



 
 
 
23
.
0
2
.
0
0
0
53.33
0.7556
2.667
0
53.10
0.5556
2.667
0

















 23
.
0
2
.
0
0
0
33
.
58
8264
.
0
917
.
2
0
2
.
279
1
12
144



Divide Equation 2 by 2.917 and
multiply it by 2.667,
.
9143
.
0
917
.
2
667
.
2

Subtract the result from
Equation 3
Substitute new equation for
Equation 3

54. Back Substitution

55. Back Substitution
1.15
2
.
0
23
.
0
23
.
0
2
.
0
3
3







a
a
Solving for a3













































 23
0
33
58
2
279
2
0
0
0
8264
0
917
2
0
1
12
144
23
.
0
2
.
0
0
0
33
.
58
8264
.
0
917
.
2
0
2
.
279
1
12
144
3
2
1
.
.
.
a
a
a
.
.
.




56. Back Substitution (cont.)
Solving for a2
67
19.
917
.
2
15
.
1
8264
.
0
33
.
58
917
.
2
8264
.
0
33
.
58
33
.
58
8264
.
0
917
.
2
3
2
3
2








a
a
a
a
































 23
0
33
58
2
279
2
0
0
0
8264
0
917
2
0
1
12
144
3
2
1
.
.
.
a
a
a
.
.
.

57. Back Substitution (cont.)
Solving for a1
2917
.
0
144
15
.
1
67
.
19
12
2
.
279
144
12
2
.
279
2
.
279
12
144
3
2
1
3
2
1











a
a
a
a
a
a
































 23
0
33
58
2
279
2
0
0
0
8264
0
917
2
0
1
12
144
3
2
1
.
.
.
a
a
a
.
.
.

58. Gaussian Elimination with Partial
Pivoting Solution































2
279
2
177
8
106
1
12
144
1
8
64
1
5
25
3
2
1
.
.
.
a
a
a





















15
.
1
67
.
19
2917
.
0
3
2
1
a
a
a

59. Topic3 Saed Sasi 59
Lecture 16
Tridiagonal, and Banded Systems
 Tridiagonal Systems
 Diagonal Dominance
 Tridiagonal Algorithm
 Examples

60. Topic3 Saed Sasi 60
Tridiagonal Systems:
 The non-zero elements are
in the main diagonal,
super diagonal and
subdiagonal.
 aij=0 if |i-j| > 1

















































5
4
3
2
1
5
4
3
2
1
6
1
0
0
0
1
4
1
0
0
0
2
6
2
0
0
0
1
4
3
0
0
0
1
5
b
b
b
b
b
x
x
x
x
x
Tridiagonal Systems

61. Topic3 Saed Sasi 61
 Occur in many applications
 Needs less storage (4n-2 compared to n2 +n for the general cases)
 (Elements of A + elements of b ; 13+5=18)
 Selection of pivoting rows is unnecessary
(under some conditions)
 Efficiently solved by Gaussian elimination
Tridiagonal Systems

62. Topic3 Saed Sasi 62
 Based on Naive Gaussian elimination.
 As in previous Gaussian elimination algorithms
 Forward elimination step
 Backward substitution step
 Elements in the super diagonal are not affected.
 Elements in the main diagonal, and B need
updating
Algorithm to Solve Tridiagonal Systems

63. Topic3 Saed Sasi 63
Tridiagonal System






































































































n
n
n
n
n
n
n
n
n
b
b
b
b
x
x
x
x
d
c
d
c
d
c
d
b
b
b
b
x
x
x
x
d
a
c
d
a
c
d
a
c
d
b
a, d








 3
2
1
3
2
1
1
3
2
2
1
1
3
2
1
3
2
1
1
1
3
2
2
2
1
1
1
and
and
d
update
to
need
We

64. Topic3 Saed Sasi 64
Diagonal Dominance
row.
ing
correspond
in the
elements
of
sum
the
n
larger tha
is
element
diagonal
each
of
magnitude
The
)
1
(
a
a
if
dominant
diagonally
is
matrix
A
,
1
ij
ii n
i
for
A
n
i
j
j


 



65. Topic3 Saed Sasi 65
Diagonal Dominance
dominant
Diagonally
Not
dominant
Diagonally
1
2
1
2
3
2
1
0
3
5
2
1
1
6
1
1
0
3
:
Examples






















66. Topic3 Saed Sasi 66
Diagonally Dominant Tridiagonal System
 A tridiagonal system is diagonally dominant if
(i.e., |diagonal element| > sum of |element before| and |element after| diagonal element)
 Forward Elimination preserves diagonal dominance
)
1
(
1 n
i
a
c
d i
i
i 


 

67. Topic3 Saed Sasi 67
Solving Tridiagonal System
  1
,...,
2
,
1
for
1
on
Substituti
Backward
2
n
Eliminatio
Forward
1
1
1
1
1
1
1



































n
n
i
x
c
b
d
x
d
b
x
n
i
b
d
a
b
b
c
d
a
d
d
i
i
i
i
i
n
n
n
i
i
i
i
i
i
i
i
i
i






































































































n
n
n
n
n
n
n
n
n
b
b
b
b
x
x
x
x
d
c
d
c
d
c
d
b
b
b
b
x
x
x
x
d
a
c
d
a
c
d
a
c
d
b
a, d








 3
2
1
3
2
1
1
3
2
2
1
1
3
2
1
3
2
1
1
1
3
2
2
2
1
1
1
and
and
d
update
to
need
We

68. Topic3 Saed Sasi 68
Example
  1
,
2
,
3
for
1
,
on
Substituti
Backward
4
2
,
n
Eliminatio
Forward
6
8
9
12
,
2
2
2
,
1
1
1
,
5
5
5
5
6
8
9
12
5
1
2
5
1
2
5
1
2
5
Solve
1
1
1
1
1
1
1
4
3
2
1



























































































































i
x
c
b
d
x
d
b
x
i
b
d
a
b
b
c
d
a
d
d
B
C
A
D
x
x
x
x
i
i
i
i
i
n
n
n
i
i
i
i
i
i
i
i
i
i

69. Topic3 Saed Sasi 69
Example
5619
.
4
5652
.
4
5652
.
6
1
6
,
5619
.
4
5652
.
4
2
1
5
5652
.
6
6
.
4
6
.
6
1
8
,
5652
.
4
6
.
4
2
1
5
6
.
6
5
12
1
9
,
6
.
4
5
2
1
5
n
Eliminatio
Forward
6
8
9
12
,
2
2
2
,
1
1
1
,
5
5
5
5
3
3
3
4
4
3
3
3
4
4
2
2
2
3
3
2
2
2
3
3
1
1
1
2
2
1
1
1
2
2








































































































































b
d
a
b
b
c
d
a
d
d
b
d
a
b
b
c
d
a
d
d
b
d
a
b
b
c
d
a
d
d
B
C
A
D

70. Topic3 Saed Sasi 70
Example
Backward Substitution
 After the Forward Elimination:
 Backward Substitution:
   
2
5
1
2
12
1
6
.
4
1
2
6
.
6
1
5652
.
4
1
2
5652
.
6
,
1
5619
.
4
5619
.
4
5619
.
4
5652
.
6
6
.
6
12
,
5619
.
4
5652
.
4
6
.
4
5
1
2
1
1
1
2
3
2
2
2
3
4
3
3
3
4
4
4























d
x
c
b
x
d
x
c
b
x
d
x
c
b
x
d
b
x
B
D T
T

71. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.
Elimination methods such as Gaussian Elimination and LU
Decomposition are prone to prone to round-off error.
Also: If the physics of the problem are understood, a close initial
guess can be made, decreasing the number of iterations needed.

72. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
1
1
3
13
2
12
1
11 ... b
x
a
x
a
x
a
x
a n
n 




2
3
23
2
22
1
21 ... b
x
a
x
a
x
a
x
a n
2n 




n
n
nn
n
n
n b
x
a
x
a
x
a
x
a 



 ...
3
3
2
2
1
1
. .
. .
. .
If: the diagonal elements are
non-zero
Rewrite each equation solving
for the corresponding unknown
ex:
First equation, solve for x1
Second equation, solve for x2

73. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
Rewriting each equation
11
1
3
13
2
12
1
1
a
x
a
x
a
x
a
c
x n
n






nn
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
a
x
a
x
a
x
a
c
x
a
x
a
x
a
x
a
x
a
c
x
a
x
a
x
a
x
a
c
x
1
1
,
2
2
1
1
1
,
1
,
1
2
2
,
1
2
2
,
1
1
1
,
1
1
1
22
2
3
23
1
21
2
2



































From Equation 1
From equation 2
From equation n-1
From equation n

74. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
General Form of each equation
11
1
1
1
1
1
a
x
a
c
x
n
j
j
j
j





22
2
1
2
2
2
a
x
a
c
x
j
n
j
j
j





1
,
1
1
1
,
1
1
1











n
n
n
n
j
j
j
j
n
n
n
a
x
a
c
x
nn
n
n
j
j
j
nj
n
n
a
x
a
c
x





1

75. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
.
,
,
2
,
1
,
1
n
i
a
x
a
c
x
ii
n
i
j
j
j
ij
i
i 






How or where can this equation be used?

76. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]
















n
-
n
2
x
x
x
x
1
1

Use rewritten equations to solve for
each value of xi.
Important: Remember to use the
most recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.

77. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
100



 new
i
old
i
new
i
i
a
x
x
x
So when has the answer been found?
The iterations are stopped when the absolute relative
approximate error is less than a prespecified tolerance for all
unknowns.

78. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
The upward velocity of a rocket
is given at three different times
Time, Velocity
5 106.8
8 177.2
12 279.2
The velocity data is approximated by a polynomial as:
  12.
t
5
,
3
2
2
1 



 a
t
a
t
a
t
v
 
s
t  
m/s
v
Table 1 Velocity vs. Time data.

79. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1































3
2
1
3
2
3
2
2
2
1
2
1
1
1
1
v
v
v
a
a
a
t
t
t
t
t
t
3
2
1
Using a Matrix template of the form
( Quadratic interpolation)
The system of equations becomes































2
.
279
2
.
177
8
.
106
1
12
144
1
8
64
1
5
25
3
2
1
a
a
a
Initial Guess: Assume an initial guess of





















5
2
1
3
2
1
a
a
a

80. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Rewriting each equation































2
.
279
2
.
177
8
.
106
1
12
144
1
8
64
1
5
25
3
2
1
a
a
a
25
5
8
.
106 3
2
1
a
a
a



8
64
2
.
177 3
1
2
a
a
a



1
12
144
2
.
279 2
1
3
a
a
a




81. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Applying the initial guess and solving for ai





















5
2
1
3
2
1
a
a
a 6720
.
3
25
)
5
(
)
2
(
5
8
.
106
a1 



    8510
.
7
8
5
6720
.
3
64
2
.
177
a2 




    36
.
155
1
8510
.
7
12
6720
.
3
144
2
.
279
a3 





Initial Guess
When solving for a2, how many of the initial guess values were used?

82. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
%
76
.
72
100
6720
.
3
0000
.
1
6720
.
3
1
a 


 x
%
47
.
125
100
8510
.
7
0000
.
2
8510
.
7
2
a 




 x
%
22
.
103
100
36
.
155
0000
.
5
36
.
155
3
a 




 x
Finding the absolute relative approximate error
100



 new
i
old
i
new
i
i
a
x
x
x At the end of the first iteration
The maximum absolute
relative approximate error is
125.47%























36
.
155
8510
.
7
6720
.
3
3
2
1
a
a
a

83. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Iteration #2
Using























36
.
155
8510
.
7
6720
.
3
3
2
1
a
a
a
  056
.
12
25
36
.
155
8510
.
7
5
8
.
106
1 




a
  882
.
54
8
36
.
155
056
.
12
64
2
.
177
2 




a
    34
.
798
1
882
.
54
12
056
.
12
144
2
.
279
3 





a
from iteration #1
the values of ai are found:

84. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error
%
543
.
69
100
056
.
12
6720
.
3
056
.
12
1
a 


 x
  %
695
.
85
100
x
882
.
54
8510
.
7
882
.
54
2






a
  %
540
.
80
100
34
.
798
36
.
155
34
.
798
3
a 





 x
At the end of the second iteration























54
.
798
882
.
54
056
.
12
3
2
1
a
a
a
The maximum absolute
relative approximate error is
85.695%

85. Iteration a1 a2 a3
1
2
3
4
5
6
3.6720
12.056
47.182
193.33
800.53
3322.6
72.767
69.543
74.447
75.595
75.850
75.906
−7.8510
−54.882
−255.51
−1093.4
−4577.2
−19049
125.47
85.695
78.521
76.632
76.112
75.972
−155.36
−798.34
−3448.9
−14440
−60072
−249580
103.22
80.540
76.852
76.116
75.963
75.931
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1





















0857
.
1
690
.
19
29048
.
0
a
a
a
3
2
1
Repeating more iterations, the following values are obtained
%
1
a
 %
2
a
 %
3
a

Notice – The relative errors are not decreasing at any significant rate
Also, the solution is not converging to the true solution of

86. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.
Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:




n
j
j
ij
a
a
i
1
ii 



n
i
j
j
ij
ii a
a
1
for all ‘i’ and for at least one ‘i’

87. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Pitfall
 











1
16
123
1
43
45
34
81
.
5
2
A
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.











129
34
96
5
53
23
56
34
124
]
B
[
Which coefficient matrix is diagonally dominant?
Most physical systems do result in simultaneous linear equations that
have diagonally dominant coefficient matrices.

88. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Given the system of equations
1
5
3
12 3
2
1 x
-
x
x 

28
3
5 3
2
1 x
x
x 


76
13
7
3 3
2
1 

 x
x
x





















1
0
1
3
2
1
x
x
x
With an initial guess of
The coefficient matrix is:
 









 

13
7
3
3
5
1
5
3
12
A
Will the solution converge using the
Gauss-Siedel method?

89. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
 









 

13
7
3
3
5
1
5
3
12
A
Checking if the coefficient matrix is diagonally dominant
4
3
1
5
5 23
21
22 





 a
a
a
10
7
3
13
13 32
31
33 





 a
a
a
8
5
3
12
12 13
12
11 






 a
a
a
The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

90. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2






























 
76
28
1
13
7
3
3
5
1
5
3
12
3
2
1
a
a
a
Rewriting each equation
12
5
3
1 3
2
1
x
x
x



5
3
28 3
1
2
x
x
x



13
7
3
76 2
1
3
x
x
x



With an initial guess of





















1
0
1
3
2
1
x
x
x
    50000
.
0
12
1
5
0
3
1
1 



x
    9000
.
4
5
1
3
5
.
0
28
2 



x
    0923
.
3
13
9000
.
4
7
50000
.
0
3
76
3 



x

91. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
The absolute relative approximate error
%
00
.
100
100
50000
.
0
0000
.
1
50000
.
0
1




a
%
00
.
100
100
9000
.
4
0
9000
.
4
2
a 




%
662
.
67
100
0923
.
3
0000
.
1
0923
.
3
3
a 




The maximum absolute relative error after the first iteration is 100%

92. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2





















8118
.
3
7153
.
3
14679
.
0
3
2
1
x
x
x
After Iteration #1
    14679
.
0
12
0923
.
3
5
9000
.
4
3
1
1 



x
    7153
.
3
5
0923
.
3
3
14679
.
0
28
2 



x
    8118
.
3
13
900
.
4
7
14679
.
0
3
76
3 



x
Substituting the x values into the
equations
After Iteration #2





















0923
.
3
9000
.
4
5000
.
0
3
2
1
x
x
x

93. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Iteration #2 absolute relative approximate error
%
61
.
240
100
14679
.
0
50000
.
0
14679
.
0
1
a 




%
889
.
31
100
7153
.
3
9000
.
4
7153
.
3
2
a 




%
874
.
18
100
8118
.
3
0923
.
3
8118
.
3
3
a 




The maximum absolute relative error after the first iteration is 240.61%
This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?

94. Iteration a1 a2 a3
1
2
3
4
5
6
0.50000
0.14679
0.74275
0.94675
0.99177
0.99919
100.00
240.61
80.236
21.546
4.5391
0.74307
4.9000
3.7153
3.1644
3.0281
3.0034
3.0001
100.00
31.889
17.408
4.4996
0.82499
0.10856
3.0923
3.8118
3.9708
3.9971
4.0001
4.0001
67.662
18.876
4.0042
0.65772
0.074383
0.00101
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
%
1
a
 %
2
a
 %
3
a






















4
3
1
3
2
1
x
x
x





















0001
.
4
0001
.
3
99919
.
0
3
2
1
x
x
x
The solution obtained is close to the exact solution of .

95. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 3
Given the system of equations
76
13
7
3 3
2
1 

 x
x
x
28
3
5 3
2
1 

 x
x
x
1
5
3
12 3
2
1 

 x
x
x
With an initial guess of





















1
0
1
3
2
1
x
x
x
Rewriting the equations
3
13
7
76 3
2
1
x
x
x



5
3
28 3
1
2
x
x
x



5
3
12
1 2
1
3




x
x
x

96. Iteration a1 A2 a3
1
2
3
4
5
6
21.000
−196.15
−1995.0
−20149
2.0364×105
−2.0579×105
95.238
110.71
109.83
109.90
109.89
109.89
0.80000
14.421
−116.02
1204.6
−12140
1.2272×105
100.00
94.453
112.43
109.63
109.92
109.89
50.680
−462.30
4718.1
−47636
4.8144×105
−4.8653×106
98.027
110.96
109.80
109.90
109.89
109.89
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 3
Conducting six iterations, the following values are obtained
%
1
a
 %
2
a
 %
3
a

The values are not converging.
Does this mean that the Gauss-Seidel method cannot be used?

97. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
 












5
3
12
3
5
1
13
7
3
A
But this is the same set of
equations used in example #2,
which did converge.
 









 

13
7
3
3
5
1
5
3
12
A
If a system of linear equations is not diagonally dominant, check to see if
rearranging the equations can form a diagonally dominant matrix.

98. http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations
3
3
2
1 

 x
x
x
9
4
3
2 3
2
1 

 x
x
x
9
7 3
2
1 

 x
x
x
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?