This document discusses methods for solving systems of linear equations, including Gauss elimination and iterative Gauss methods like the Gauss-Jacobi and Gauss-Seidel methods. It provides explanations of row-echelon form and reduced row-echelon form. Examples are given to demonstrate using elementary row operations to solve systems of linear equations and the Gauss-Jacobi iterative method.

2.  Basics
 Gauss elimination method
 Gauss iterative method:
Gauss Jacobi method
Gauss sidle method

3. mnmnm
nn
nn
nn
bxaxbx
bxaxbx
bxaxbx
bxaxbx




*.......**a
.
.
.
*..........**a
*..........**a
*..........**a
211m1
33232131
22222121
11211111

4.  This system consists of linear equations, each with
coefficients, and has unknowns which have to fulfill
the set of equations simultaneously. To simplify
notation, it is possible to rewrite the above equations
in matrix notation:































nnmnmm
n
n
b
b
b
x
x
x
aaa
aaa
aaa
2
1
2
1
.........21
22212
11211
......
.......
Simply ,we can say BAX 

5.  Where A is called matrix of coefficient of order m*n
......
.......
.........21
22212
11211










mnmm
n
n
aaa
aaa
aaa
 X is a simple vector of order B is a simple vector of order










nx
x
x
2
1










nb
b
b
2
1

6.  We can write a system of linear equations as a matrix by
writing only the coefficients and constants that appear in the
equations.
This is called the augmented matrix of the system.
mmnmm
n
n
baaa
baaa
baaa
21
222221
111211

7. To solve linear systems correspond to operations on
the rows of the augmented matrix of the system.
 For example, adding a multiple of one equation
to another corresponds to adding a multiple of
one row to another.

8. For example,
 Add a multiple of one row to another.
 Multiply a row by a nonzero constant.
 Interchange two rows.
“Note that performing any of these operations on the
augmented matrix of a system does not change its
solution.”

9. We use the following notation to describe
the elementary row operations:
Symbol Description
Ri + kRj → Ri
Change the ith row by adding
k times row j to it.
Then, put the result back in row i.
kRi Multiply the ith row by k.
Ri ↔ Rj Interchange the ith and jth rows.

11. In general, to solve a system of linear equations using
its augmented matrix, we use elementary row
operations to arrive at a matrix in a certain form.
This form is described as follows.

12.  A matrix is in row-echelon form if it satisfies the
following conditions:
 The first nonzero number in each row
(reading from left to right) is 1. This is called the
leading entry.
 The leading entry in each row is to the right of
the leading entry in the row immediately above
it.
 All rows consisting entirely of zeros are at
the bottom of the matrix.

13.  A matrix is in reduced row-echelon form if it is in
row-echelon form and also satisfies the following
condition.
 Every number above and below each leading entry
is a 0.

14. 1
2
Leading 1's do not
shift to the right
in successive rows.
0 0 7
0 3 4 5
0 0 0 0.4
1
1
1
10 1 0 0
 
  
 
 
 
  
Not in row-echelon form:

15. Row-Echelon Form Reduced
Row-Echelon Form
   
       
   
   
   
     
1 1
2 2
Leading 1's haveLeading 1's shift to
0's above andthe right in
below them.successive rows.
3 6 10 0 3 0 0 0
0 0 4 3 0 0 0 3
0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
1 1
1 1
1 1

16. Start by obtaining 1 in the top left corner.
Then, obtain zeros below that 1 by adding
appropriate multiples of the first row to
the rows below it.
1
0
0
 
 
 
  

17. We now discuss a systematic way
to put a matrix in row-echelon form
using elementary row operations.
 We see how the process might work
for a 3 x 4 matrix.

18. Next, obtain a leading 1 in the next row.
Then, obtain zeros below that 1.
 At each stage, make sure every leading entry is
to the right of the leading entry in the row above it.
 Rearrange the rows if necessary.
1 1
0 0 1
0 0 0
   
      
      

19. Solve the system of linear equations
using Gaussian elimination.
 We first write the augmented matrix of the system.
 Then, we use elementary row operations to put it
in row-echelon form.
4 8 4 4
3 8 5 11
2 12 17
x y z
x y z
x y z
  

   
     

20. 1
14
R
4 8 4 4
3 8 5 11
2 1 12 17
2 1 1
3 8 5 11
2 1
1
12 17

 
  
   
 
  
   

21. 2 1 2
3 1 3
1
22
R 3R R
R 2R R
R
2 1 1
2 8 14
5 1
1
0
0
1
0 15
2 1 1
4 7
5 10 15
0 1
0
 
 
 
  
 


 
  
 


 

22.  3 2 3
1
310
R 5R R
R
1
0 1
0 0
2 1 1
4 7
10 20
2 1 1
4 7
2
1
0 1
0 0 1
 

 
  
  
 
  
 







23. We now have an equivalent matrix in row-echelon form.
The corresponding system of equations is:
 We use back-substitution to solve the system.
2 1
4 7
2
x y z
y z
z
  

  
  

24.  y + 4(–2) = –7
y = 1
 x + 2(1) – (–2) = 1
x = –3
The solution of the system is:(–3, 1, –2)

25.  Let the set of equations given be:
Where the element of the diagonal are the largest in
each column.
We can get the values of unknowns as follows:
mnmnm
nn
nn
nn
bxaxbx
bxaxbx
bxaxbx
bxaxbx




*.......**a
.
.
.
*..........**a
*..........**a
*..........**a
211m1
33232131
22222121
11211111

26.  Rewrite the equations as:
 Then make initial approximation of the solution
x1,x2,..xn.
)(
1 0
1
0
2121
11
1
1 nn xaxab
a
x  
)(
1 0
2
0
323
0
1212
22
1
2 nn xaxaxab
a
x  
)(
1 0
11
0
22
0
11
1
 nnnnnn
nn
n xaxaxab
a
x 

27. 27









n
ji
j
ijii
n
ji
j ii
ij
aa
a
a
E
1
1
11 n1,2,...,ifor
Evaluate the infinity(maximum row sum) norm of E
Diagonally dominant matrix
If A is a diagonally dominant matrix, then Jacobi iteration converges for any
initial vector

28. 28
Example 1 (Jacobi Iteration)


































15
21
7
512
184
114
3
2
1
x
x
x











0
0
0
0
x
5
215
8
421
4
7
0
2
0
11
3
0
3
0
11
2
0
3
0
21
1
xx
x
xx
x
xx
x






0.3
5
15
625.2
8
21
75.1
4
7



7395.26
2
0
 Axb
0452.10
2
1
 Axb
Diagonally dominant matrix

29. 29
Example 1 continued...
5
215
8
421
4
7
1
2
1
12
3
1
3
1
12
2
1
3
1
22
1
xx
x
xx
x
xx
x






225.4
5
625.275.1215
875.3
8
375.1421
65625.1
4
3625.27









7413.6
2
2
 Axb
8875.2
5
875.365625.1215
98125.3
8
225.465625.1421
6625.1
4
225.4875.37
3
3
3
2
3
1









x
x
x
9534.1
2
2
 Axb
Matrix is diagonally dominant, Jacobi iterations are converging

30. 30
Example 2


































7
21
15
114
184
512
3
2
1
x
x
x











0
0
0
0
x 7395.26
2
0
 Axb
0
2
0
1
1
3
0
3
0
11
2
0
3
0
21
1
47
8
421
2
515
xxx
xx
x
xx
x





0.7
625.2
8
21
5.7
2
15





The matrix is not diagonally dominant
8546.54
2
1
 Axb

31. 31
Example 2 continued...
625.39625.25.747
25.0
8
75.7421
3125.11
2
75625.215
1
3
1
2
1
1







x
x
x
3761.208
2
2
 Axb
The residual term is increasing at each iteration, so the iterations are diverging.
Note that the matrix is not diagonally dominant

32.  Like for Gauss Jacobi we’ll first ensure that the elements of the diagonal
are the maximum values
 Then solve the 1s equation for the 1st unknown, 2nd for the 2nd
unknown, etc.
First iteration:
i. Find the value of x1 by substituting the initial values of other
unknowns
ii. Find the value of x2 by substituting the current value of x1 and initial
values of other unknowns.
and continue......
Second iteration:
i. Substitute all the values obtained in 1st iteration and find x1
ii. Use the current values of other unknowns, find x2
and continue....

33. 33
Convergence of Gauss-Seidel
iteration
 GS iteration converges for any initial vector if A is a
diagonally dominant matrix
 GS iteration converges for any initial vector if A is a
symmetric and positive definite matrix
 Matrix A is positive definite if
xTAx>0 for every nonzero x vector

34. Given the system of equations
15312 321 x-xx 
2835 321 xxx 
761373 321  xxx





















1
0
1
3
2
1
x
x
x
With an initial guess of
The coefficient matrix is:
 









 

1373
351
5312
A
Will the solution converge using the
Gauss-Siedel method?

35.  









 

1373
351
5312
A
Checking if the coefficient matrix is
diagonally dominant
43155 232122  aaa
10731313 323133  aaa
8531212 131211  aaa
The inequalities are all true and at least one row is strictly
greater than, Therefore: The solution should converge
using the Gauss-Siedel Method

36. 





























 
76
28
1
1373
351
5312
3
2
1
a
a
a





















1
0
1
3
2
1
x
x
x
Rewriting each equation
12
531 32
1
xx
x


5
328 31
2
xx
x


13
7376 21
3
xx
x


With an initial guess of
    50000.0
12
15031
1 

x
    9000.4
5
135.028
2 

x
    0923.3
13
9000.4750000.0376
3 

x

37. 




















8118.3
7153.3
14679.0
3
2
1
x
x
x





















0923.3
9000.4
5000.0
3
2
1
x
x
x
After Iteration #1
    14679.0
12
0923.359000.431
1 

x
    7153.3
5
0923.3314679.028
2 

x
    8118.3
13
900.4714679.0376
3 

x
Substituting the x values into the
equations
After Iteration #2

38. Iteration a1 a2 a3
1
2
3
4
5
6
0.50000
0.14679
0.74275
0.94675
0.99177
0.99919
4.9000
3.7153
3.1644
3.0281
3.0034
3.0001
3.0923
3.8118
3.9708
3.9971
4.0001
4.0001
Repeating more iterations, the following values are
obtained





















4
3
1
3
2
1
x
x
x





















0001.4
0001.3
99919.0
3
2
1
x
x
x
The solution obtained is close to the exact solution of
.