NUMERICAL METHOD

1. 1
Numerical Methods
Solution of Systems of Linear Equations

2. 2
Vector, Matrices, and
Linear Equations

3. 3
VECTORS
 


























































1
0
0
0
,
0
1
0
0
,
0
0
1
0
,
0
0
0
1
1
2
tor
column vec
2
4
1
vector
row
:
Examples
numbers
of
array
l
dimensiona
one
a
:
Vector
4
3
2
1 e
e
e
e
vectors
Identity

4. 4
MATRICES




































1
2
0
0
1
4
1
0
0
1
4
3
0
0
2
1
l
Tridiagona
,
6
0
0
0
0
0
0
0
0
0
4
0
0
0
0
1
diagonal
1
0
0
1
matrix
identity
0
0
0
0
0
0
matrix
zero
:
Examples
numbers
of
array
l
dimensiona
two
a
:
Matrix

5. 5
MATRICES
























1
0
0
0
1
4
0
0
0
1
4
0
3
1
2
1
ngular
upper tria
,
4
5
1
5
0
1
1
1
2
symmetric
:
Examples

6. 6
Determinant of a MATRICES
82
)
0
15
(
1
)
5
12
(
1
)
25
(
2
5
0
1
-
3
1
-
4
5
1
-
3
1
-
4
5
5
0
2
4
5
1
5
0
1
1
3
2
det
:
Examples
only
matrices
square
for
Defined






















7. 7
Adding and Multiplying Matrices
j
i
j
i
m
k
,
b
a
c
B
A
C
*
p
m
if
only
defined
is
AB
C
product
The
*
q)
B(p
and
m)
(n
A
matrices
two
of
tion
Multiplica
,
b
a
c
B
A
C
*
size
same
the
have
they
if
only
Defined
*
B
and
A
matrices
two
of
addition
The
1
kj
ik
ij
ij
ij
ij

















8. 8
Systems of Linear Equations
form
Matrix
form
Standard
7
5
3
6
0
1
3
1
5
.
2
3
4
2
7
6
5
3
5
.
2
3
3
4
2
forms
different
in
presented
be
can
equations
linear
of
system
A
3
2
1
3
1
3
2
1
3
2
1
















































x
x
x
x
x
x
x
x
x
x
x

9. 9
Solutions of Linear Equations
5
2
3
:
equations
following
the
o
solution t
a
is
2
1
2
1
2
1
2
1

















x
x
x
x
x
x

10. 10
Solutions of Linear Equations
 A set of equations is inconsistent if there
exists no solution to the system of equations:
nt
inconsiste
are
equations
These
5
4
2
3
2
2
1
2
1




x
x
x
x

11. 11
Solutions of Linear Equations
 Some systems of equations may have infinite
number of solutions
all
for
solution
a
is
)
3
(
5
.
0
solutions
of
number
infinite
have
6
4
2
3
2
2
1
2
1
2
1
a
a
a
x
x
x
x
x
x



















12. 12
Graphical Solution of Systems of
Linear Equations
5
2
3
2
1
2
1




x
x
x
x
Solution
x1=1, x2=2

13. 13
Cramer’s Rule is Not Practical
way
efficient
in
computed
are
ts
determinan
the
if
used
be
can
It
needed.
are
tions
multiplica
10
2.38
system,
30
by
30
a
solve
To
tions.
multiplica
1)N!
-
1)(N
(N
requires
system
N
by
N
solve
To
.
systems
large
for
practical
not
is
Rule
s
Cramer'
2
2
1
1
1
5
1
3
1
,
1
2
1
1
1
2
5
1
3
system
the
solve
to
used
be
can
Rule
s
Cramer'
35
2
1





 x
x

14. 14
 Naive Gaussian Elimination
 Examples
Lecture 13
Naive Gaussian Elimination

15. 15
Naive Gaussian Elimination
 The method consists of two steps:
 Forward Elimination: the system is
reduced to upper triangular form. A sequence
of elementary operations is used.
 Backward Substitution: Solve the system
starting from the last variable.































































'
'
'
0
0
'
'
0
3
2
1
3
2
1
33
23
22
13
12
11
3
2
1
3
2
1
33
32
31
23
22
21
13
12
11
b
b
b
x
x
x
a
a
a
a
a
a
b
b
b
x
x
x
a
a
a
a
a
a
a
a
a

16. 16
Elementary Row Operations
 Adding a multiple of one row to another
 Multiply any row by a non-zero constant

17. 17
Example
Forward Elimination
























































































18
27
6
16
14
3
2
0
1
8
12
0
2
2
4
0
4
2
2
6
4
3,
2,
equations
from
Eliminate
:
Step1
n
Eliminatio
Forward
:
1
Part
34
19
26
16
18
1
4
6
3
9
13
3
10
6
8
12
4
2
2
6
4
3
2
1
1
4
3
2
1
x
x
x
x
x
x
x
x
x

18. 18
Example
Forward Elimination
























































































3
9
6
16
3
0
0
0
5
2
0
0
2
2
4
0
4
2
2
6
4
equation
from
Eliminate
:
Step3
21
9
6
16
13
4
0
0
5
2
0
0
2
2
4
0
4
2
2
6
4
,
3
equations
from
Eliminate
:
Step2
4
3
2
1
3
4
3
2
1
2
x
x
x
x
x
x
x
x
x
x

19. 19
Example
Forward Elimination

























































































3
9
6
16
3
0
0
0
5
2
0
0
2
2
4
0
4
2
2
6
34
19
26
16
18
1
4
6
3
9
13
3
10
6
8
12
4
2
2
6
:
n
Eliminatio
Forward
the
of
Summary
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x

20. 20
Example
Backward Substitution
3
6
)
1
(
4
)
2
(
2
)
1
(
2
16
,
1
4
)
1
(
2
)
2
(
2
6
2
2
5
9
,
1
3
3
for
solve
,...
for
solve
then
,
for
Solve
3
9
6
16
3
0
0
0
5
2
0
0
2
2
4
0
4
2
2
6
1
2
3
4
1
3
4
4
3
2
1


































































x
x
x
x
x
x
x
x
x
x
x

21. 21
Forward Elimination
n
i
b
a
a
b
b
n
j
a
a
a
a
a
x
n
i
b
a
a
b
b
n
j
a
a
a
a
a
x
i
i
i
j
i
ij
ij
i
i
i
j
i
ij
ij






















































3
)
2
(
eliminate
To
2
)
1
(
eliminate
To
2
22
2
2
22
2
2
1
11
1
1
11
1
1

22. 22
Forward Elimination
.
eliminated
is
until
Continue
1
)
(
eliminate
To
1





























n
k
kk
ik
i
i
kj
kk
ik
ij
ij
k
x
n
i
k
b
a
a
b
b
n
j
k
a
a
a
a
a
x

23. 23
Backward Substitution
i
i
n
i
j
j
j
i
i
i
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
a
x
a
b
x
a
x
a
x
a
b
x
a
x
a
b
x
a
b
x
,
1
,
2
,
2
1
1
,
2
,
2
2
2
1
,
1
,
1
1
1
,

























24. 24
 Summary of the Naive Gaussian Elimination
 Example
 Problems with Naive Gaussian Elimination
 Failure due to zero pivot element
 Error
 Pseudo-Code
Naive Gaussian Elimination

25. 25
Naive Gaussian Elimination
o The method consists of two steps
o Forward Elimination: the system is reduced to
upper triangular form. A sequence of elementary
operations is used.
o Backward Substitution: Solve the system starting
from the last variable. Solve for xn ,xn-1,…x1.































































'
'
'
0
0
'
'
0
3
2
1
3
2
1
33
23
22
13
12
11
3
2
1
3
2
1
33
32
31
23
22
21
13
12
11
b
b
b
x
x
x
a
a
a
a
a
a
b
b
b
x
x
x
a
a
a
a
a
a
a
a
a

26. 26
Example 1
17
7
5
6
4
8
3
2
1
1
3
3
3
7
2
3
1
1
2
2
2
10
2
3
2
)
(
1
8
3
2
3
,
2
equations
from
Eliminate
:
Step1
___
n
Eliminatio
Forward
:
1
Part
:
n
Eliminatio
Gaussian
Naive
using
Solve
3
2
3
2
3
2
1
3
2
1
3
2
1
3
2
1
1




































x
x
x
x
x
x
x
eq
eq
eq
x
x
x
eq
eq
eq
x
x
x
equation
pivot
unchanged
eq
x
x
x
x

27. 27
Example 1



































13
13
6
4
8
3
2
2
1
5
3
3
17
7
5
)
(
2
6
4
1
8
3
2
3
equation
from
Eliminate
:
Step2
n
Eliminatio
Forward
:
1
Part
3
3
2
3
2
1
3
2
3
2
3
2
1
2
x
x
x
x
x
x
eq
eq
eq
x
x
equation
pivot
unchanged
eq
x
x
unchanged
eq
x
x
x
x

28. 28
Example 1
Backward Substitution






































1
2
1
is
solution
The
1
3
2
8
2
1
4
6
1
13
13
3
2
1
1
,
1
3
2
1
,
1
3
3
,
1
2
2
,
1
1
1
3
2
,
2
3
3
,
2
2
2
3
,
3
3
3
x
x
x
a
x
x
a
x
a
x
a
b
x
x
a
x
a
b
x
a
b
x

29. 29
Determinant
13
det
det
13
0
0
4
1
0
3
2
1
A'
2
1
3
2
3
2
3
2
1
A
:
Example
t
determinan
affect the
not
do
operations
elementary
The
operations
Elementary





















 












(A')
(A)

30. 30
How Many Solutions Does a System of
Equations AX=B Have?
0
elements
0
elements
B
ing
correspond
B
ing
correspond
rows
zero
rows
zero
more
or
one
has
more
or
one
has
rows
zero
no
has
matrix
reduced
matrix
reduced
matrix
reduced
0
det(A)
0
det(A)
0
det(A)
Infinite
solution
No
Unique






31. 31
Examples







































































































5
.
1
!
1
0
5
.
0
0
#
:
0
2
0
0
2
1
1
2
0
0
2
1
1
1
2
0
2
1
4
2
4
2
2
1
3
2
4
2
2
1
2
1
4
3
2
1
solutions
of
#
infinte
solution
No
Unique
X
impossible
X
solutions
Infinite
solution
No
solution
X
X
X
X
X
X

32. Pseudo-Code: Forward Elimination
Do k = 1 to n-1
Do i = k+1 to n
factor = ai,k / ak,k
Do j = k+1 to n
ai,j = ai,j – factor * ak,j
End Do
bi = bi – factor * bk
End Do
End Do
32

33. Pseudo-Code: Back Substitution
xn = bn / an,n
Do i = n-1 downto 1
sum = bi
Do j = i+1 to n
sum = sum – ai,j * xj
End Do
xi = sum / ai,i
End Do
33

34. 34
Gaussian Elimination with
Scaled Partial Pivoting
 Problems with Naive Gaussian Elimination
 Definitions and Initial step
 Forward Elimination
 Backward substitution
 Example

35. 35
Problems with Naive Gaussian Elimination
o The Naive Gaussian Elimination may fail for
very simple cases. (The pivoting element is zero).
o Very small pivoting element may result in
serious computation errors



















2
1
1
1
1
0
2
1
x
x




















 
2
1
1
1
1
10
2
1
10
x
x

36. 36
Example 2





































 
1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
:
Pivoting
Partial
Scaled
with
on
Eliminati
Gaussian
using
system
following
the
Solve
4
3
2
1
x
x
x
x

37. 37
Example 2
Initialization step
 
 
4
3
2
1
L
Vector
Index
5
8
4
2
S
vector
Scale
1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
4
3
2
1







































 
x
x
x
x
Scale vector:
disregard sign
find largest in
magnitude in
each row

38. 38
Why Index Vector?
 Index vectors are used because it is much
easier to exchange a single index element
compared to exchanging the values of a
complete row.
 In practical problems with very large N,
exchanging the contents of rows may not
be practical.

39. 39
Example 2
Forward Elimination-- Step 1: eliminate x1
]
1
3
2
4
[
Exchange
equation
pivot
first
the
is
4
equation
to
s
correspond
max
5
4
,
8
5
,
4
3
,
2
1
4
,
3
,
2
,
1
]
4
3
2
1
[
]
5
8
4
2
[
1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
equation
pivot
the
of
Selection
1
4
4
1
,
4
3
2
1
































































 
L
l
and
l
l
i
S
a
Ratios
L
S
x
x
x
x
i
i
l
l

40. 40
Example 2
Forward Elimination-- Step 1: eliminate x1
















































































 
1
25
.
2
75
.
1
25
.
1
3
5
2
4
75
.
0
25
.
0
5
.
5
0
75
.
1
75
.
2
5
.
0
0
25
.
0
75
.
0
5
.
1
0
1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
B
and
A
Update
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
First pivot
equation

41. 41
Example 2
Forward Elimination-- Step 2: eliminate x2
]
2
3
1
4
[
2
5
.
1
8
5
.
5
4
5
.
0
4
,
3
,
2
:
Ratios
]
1
3
2
4
[
]
5
8
4
2
[
1
25
.
2
75
.
1
25
.
1
3
5
2
4
75
.
0
25
.
0
5
.
5
0
75
.
1
75
.
2
5
.
0
0
25
.
0
75
.
0
5
.
1
0
equation
pivot
second
the
of
Selection
2
,
4
3
2
1
































































L
i
S
a
L
S
x
x
x
x
i
i
l
l

42. 42
Example 2
Forward Elimination-- Step 3: eliminate x3

















































































1
9
1667
.
2
25
.
1
3
5
2
4
2
0
0
0
8333
.
1
5
.
2
0
0
25
.
0
75
.
0
5
.
1
0
]
3
2
1
4
[
1
8333
.
6
1667
.
2
25
.
1
3
5
2
4
6667
.
1
25
.
0
0
0
8333
.
1
5
.
2
0
0
25
.
0
75
.
0
5
.
1
0
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
x
Third pivot
equation

43. 43
Example 2
Backward Substitution
7.2333
4
2
5
3
1
1.1333
5
.
1
75
.
0
25
.
0
25
.
1
2.4327
5
.
2
8333
.
1
1667
.
2
,
5
.
4
2
9
]
3
2
1
4
[
1
9
1667
.
2
25
.
1
3
5
2
4
2
0
0
0
8333
.
1
5
.
2
0
0
25
.
0
75
.
0
5
.
1
0
2
3
4
1
,
2
2
,
4
3
3
,
4
4
4
,
4
4
1
3
4
2
,
1
3
3
,
1
4
4
,
1
1
2
4
3
,
2
4
4
,
2
2
3
4
,
3
3
4
4
3
2
1
1





































































x
x
x
a
x
a
x
a
x
a
b
x
x
x
a
x
a
x
a
b
x
x
a
x
a
b
x
a
b
x
L
x
x
x
x
l

44. 44
Example 3








































1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
Pivoting
Partial
Scaled
with
n
Eliminatio
Gaussian
using
sytstem
following
the
Solve
4
3
2
1
x
x
x
x

45. 45
Example 3
Initialization step
 
 
4
3
2
1
L
Vector
Index
5
8
4
2
S
vector
Scale
1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
4
3
2
1










































x
x
x
x

46. 46
Example 3
Forward Elimination-- Step 1: eliminate x1
]
1
3
2
4
[
Exchange
equation
pivot
first
the
is
4
equation
to
s
correspond
max
5
4
,
8
5
,
4
3
,
2
1
4
,
3
,
2
,
1
]
4
3
2
1
[
]
5
8
4
2
[
1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
equation
pivot
the
of
Selection
1
4
4
1
,
4
3
2
1



































































L
l
and
l
l
i
S
a
Ratios
L
S
x
x
x
x
i
i
l
l

47. 47
Example 3
Forward Elimination-- Step 1: eliminate x1




















































































1
25
.
2
75
.
1
25
.
1
3
5
2
4
75
.
0
25
.
0
5
.
10
0
75
.
1
75
.
2
5
.
0
0
25
.
0
75
.
0
5
.
1
0
1
1
1
1
3
5
2
4
3
6
8
5
4
1
3
3
1
2
1
1
B
and
A
Update
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x

48. 48
Example 3
Forward Elimination-- Step 2: eliminate x2
]
1
2
3
4
[
2
5
.
1
8
5
.
10
4
5
.
0
4
,
3
,
2
:
Ratios
]
1
3
2
4
[
]
5
8
4
2
[
1
25
.
2
75
.
1
25
.
1
3
5
2
4
75
.
0
25
.
0
5
.
10
0
75
.
1
75
.
2
5
.
0
0
25
.
0
75
.
0
5
.
1
0
equation
pivot
second
the
of
Selection
2
,
4
3
2
1

































































L
i
S
a
L
S
x
x
x
x
i
i
l
l

49. 49
Example 3
Forward Elimination-- Step 2: eliminate x2





















































































1
2.25
1.8571
0.9286
3
5
2
4
75
.
0
25
.
0
5
.
10
0
1.7143
2.7619
-
0
0
0.3571
0.7857
0
0
]
2
3
1
4
[
1
25
.
2
75
.
1
25
.
1
3
5
2
4
75
.
0
25
.
0
5
.
10
0
75
.
1
75
.
2
5
.
0
0
25
.
0
75
.
0
5
.
1
0
B
and
A
Updating
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
x

50. 50
Example 3
Forward Elimination-- Step 3: eliminate x3
]
1
2
3
4
[
2
0.7857
4
2.7619
4
,
3
:
Ratios
]
1
2
3
4
[
]
5
8
4
2
[
1
2.25
1.8571
0.9286
3
5
2
4
75
.
0
25
.
0
5
.
10
0
1.7143
2.7619
0
0
0.3571
0.7857
0
0
equation
pivot
third
the
of
Selection
3
,
4
3
2
1
































































L
i
S
a
L
S
x
x
x
x
i
i
l
l

51. 51
Example 3
Forward Elimination-- Step 3: eliminate x3





















































































1
2.25
1.8571
1.4569
3
5
2
4
75
.
0
25
.
0
5
.
10
0
1.7143
2.7619
0
0
0.8448
0
0
0
]
1
2
3
4
[
1
2.25
1.8571
0.9286
3
5
2
4
75
.
0
25
.
0
5
.
10
0
1.7143
2.7619
0
0
0.3571
0.7857
0
0
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
x

52. 52
Example 3
Backward Substitution
1.8673
4
2
5
3
1
0.3469
0.3980
2.7619
1.7143
1.8571
,
1.7245
0.8448
1.4569
]
1
2
3
4
[
1
2.25
1.8571
1.4569
3
5
2
4
75
.
0
25
.
0
5
.
10
0
1.7143
2.7619
0
0
0.8448
0
0
0
2
3
4
1
,
2
2
,
3
3
,
4
4
,
1
2
,
3
3
,
4
4
,
2
4
3
,
4
4
,
3
4
,
4
4
3
2
1
1
1
1
1
1
2
2
2
2
3
3
3
4
4




































































x
x
x
a
x
a
x
a
x
a
b
x
a
x
a
x
a
b
x
x
a
x
a
b
x
a
b
x
L
x
x
x
x
l
l
l
l
l
l
l
l
l
l
l
l
l
l

53. 53
How Do We Know If a Solution is
Good or Not
Given AX=B
X is a solution if AX-B=0
Compute the residual vector R= AX-B
Due to rounding error, R may not be zero


i
i
r
max
if
acceptable
is
solution
The

54. 54
How Good is the Solution?
















































































0.001
0.003
0.002
0.005
:
Residues
1.7245
0.3980
0.3469
1.8673
1
1
1
1
3
5
2
4
3
6
8
5
4
1
2
3
1
2
1
1
4
3
2
1
4
3
2
1
R
x
x
x
x
solution
x
x
x
x

55. 55
Remarks:
 We use index vector to avoid the need to move
the rows which may not be practical for large
problems.
 If we order the equation as in the last value of
the index vector, we have a triangular form.
 Scale vector is formed by taking maximum in
magnitude in each row.
 Scale vector does not change.
 The original matrices A and B are used in
checking the residuals.

56. 56
Tridiagonal & Banded Systems
and Gauss-Jordan Method
 Tridiagonal Systems
 Diagonal Dominance
 Tridiagonal Algorithm
 Examples
 Gauss-Jordan Algorithm

57. 57
Tridiagonal Systems:
 The non-zero elements are
in the main diagonal,
super diagonal and
subdiagonal.
 aij=0 if |i-j| > 1

















































5
4
3
2
1
5
4
3
2
1
6
1
0
0
0
1
4
1
0
0
0
2
6
2
0
0
0
1
4
3
0
0
0
1
5
b
b
b
b
b
x
x
x
x
x
Tridiagonal Systems

58. 58
 Occur in many applications
 Needs less storage (4n-2 compared to n2 +n for the general cases)
 Selection of pivoting rows is unnecessary
(under some conditions)
 Efficiently solved by Gaussian elimination
Tridiagonal Systems

59. 59
 Based on Naive Gaussian elimination.
 As in previous Gaussian elimination algorithms
 Forward elimination step
 Backward substitution step
 Elements in the super diagonal are not affected.
 Elements in the main diagonal, and B need
updating
Algorithm to Solve Tridiagonal Systems

60. 60
Tridiagonal System






































































































'
'
3
'
2
1
3
2
1
'
1
'
3
2
'
2
1
1
3
2
1
3
2
1
1
1
3
2
2
2
1
1
1
updated
not
are
elements
The
elements
and
the
update
to
need
zeros,
be
will
elements
the
All
n
n
n
n
n
n
n
n
n
b
b
b
b
x
x
x
x
d
c
d
c
d
c
d
b
b
b
b
x
x
x
x
d
a
c
d
a
c
d
a
c
d
c
b
d
a










61. 61
Diagonal Dominance
row.
ing
correspond
in the
elements
of
sum
the
n
larger tha
is
element
diagonal
each
of
magnitude
The
)
1
(
a
a
if
dominant
diagonally
is
matrix
A
,
1
ij
ii n
i
for
A
n
i
j
j


 



62. 62
Diagonal Dominance
dominant
Diagonally
Not
dominant
Diagonally
1
2
1
2
3
2
1
0
3
5
2
1
1
6
1
1
0
3
:
Examples






















63. 63
Diagonally Dominant Tridiagonal System
 A tridiagonal system is diagonally dominant if
 Forward Elimination preserves diagonal dominance
)
1
(
1 n
i
a
c
d i
i
i 


 

64. 64
Solving Tridiagonal System
  1
,...,
2
,
1
for
1
on
Substituti
Backward
2
n
Eliminatio
Forward
1
1
1
1
1
1
1



































n
n
i
x
c
b
d
x
d
b
x
n
i
b
d
a
b
b
c
d
a
d
d
i
i
i
i
i
n
n
n
i
i
i
i
i
i
i
i
i
i

65. 65
Example
  1
,
2
,
3
for
1
,
on
Substituti
Backward
4
2
,
n
Eliminatio
Forward
6
8
9
12
,
2
2
2
,
1
1
1
,
5
5
5
5
6
8
9
12
5
1
2
5
1
2
5
1
2
5
Solve
1
1
1
1
1
1
1
4
3
2
1



























































































































i
x
c
b
d
x
d
b
x
i
b
d
a
b
b
c
d
a
d
d
B
C
A
D
x
x
x
x
i
i
i
i
i
n
n
n
i
i
i
i
i
i
i
i
i
i

66. 66
Example
5619
.
4
5652
.
4
5652
.
6
1
6
,
5619
.
4
5652
.
4
2
1
5
5652
.
6
6
.
4
6
.
6
1
8
,
5652
.
4
6
.
4
2
1
5
6
.
6
5
12
1
9
,
6
.
4
5
2
1
5
n
Eliminatio
Forward
6
8
9
12
,
2
2
2
,
1
1
1
,
5
5
5
5
3
3
3
4
4
3
3
3
4
4
2
2
2
3
3
2
2
2
3
3
1
1
1
2
2
1
1
1
2
2








































































































































b
d
a
b
b
c
d
a
d
d
b
d
a
b
b
c
d
a
d
d
b
d
a
b
b
c
d
a
d
d
B
C
A
D

67. 67
Example
Backward Substitution
 After the Forward Elimination:
 Backward Substitution:
   
2
5
1
2
12
1
6
.
4
1
2
6
.
6
1
5652
.
4
1
2
5652
.
6
,
1
5619
.
4
5619
.
4
5619
.
4
5652
.
6
6
.
6
12
,
5619
.
4
5652
.
4
6
.
4
5
1
2
1
1
1
2
3
2
2
2
3
4
3
3
3
4
4
4























d
x
c
b
x
d
x
c
b
x
d
x
c
b
x
d
b
x
B
D T
T

68. 68
Gauss-Jordan Method
 The method reduces the general system of
equations AX=B to IX=B where I is an identity
matrix.
 Only Forward elimination is done and no
backward substitution is needed.
 It has the same problems as Naive Gaussian
elimination and can be modified to do partial
scaled pivoting.
 It takes 50% more time than Naive Gaussian
method.

69. 69
Gauss-Jordan Method
Example






























































































2
7
0
2
0
0
5
6
0
1
1
1
1
1
2
3
3
1
1
4
2
2
2
/
1
1
3
2
from
x
Eleminate
1
2
7
0
4
2
2
1
2
4
2
2
2
3
2
1
1
3
2
1
x
x
x
eq
eq
eq
eq
eq
eq
eq
eq
and
equations
Step
x
x
x

70. 70
Gauss-Jordan Method
Example























































 




































2
1.1667
1.1667
2
0
0
0.8333
1
0
1667
.
0
0
1
2
1
0
3
3
2
1
1
1
1
6
/
2
2
3
1
from
x
Eleminate
2
2
7
0
2
0
0
5
6
0
1
1
1
3
2
1
2
3
2
1
x
x
x
eq
eq
eq
eq
eq
eq
eq
eq
and
equations
Step
x
x
x

71. 71
Gauss-Jordan Method
Example














































 











































1
2
1
1
0
0
0
1
0
0
0
1
3
1
0.8333
2
2
3
1
1667
.
0
1
1
2
/
3
3
2
1
from
x
Eleminate
3
2
1.1667
1.1667
2
0
0
0.8333
1
0
1667
.
0
0
1
3
2
1
3
3
2
1
x
x
x
eq
eq
eq
eq
eq
eq
eq
eq
and
equations
Step
x
x
x

72. 72
Gauss-Jordan Method
Example























































































1
2
1
1
2
1
1
0
0
0
1
0
0
0
1
2
7
0
4
2
2
1
2
4
2
2
2
3
2
1
3
2
1
3
2
1
x
x
x
is
solution
x
x
x
to
d
transforme
is
x
x
x