Computational Method for Engineers: Solving a system of Linear Equations

Chapter 3
Solving System of Linear
Equations

 A system of linear Equations is composed of linear
equations that are aimed at expressing a physical
system or process using linear relationship .
 An n- number of equation with n- number of
unknowns is written
𝑓1 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0
𝑓2 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0
𝑓3 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0
𝑓𝑛 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0
SolvingSystemofLinearEquations

 A linear equation has a general form of
f(x) = a x + b = 0
 An n - number of linear equation with n- number of
unknowns is written in general form
𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3 + ⋯ + 𝑎1𝑛𝑥𝑛 = 𝑏1
𝑎21𝑥1 + 𝑎22𝑥2 + 𝑎23𝑥3 + ⋯ + 𝑎2𝑛𝑥𝑛 = 𝑏2
𝑎31𝑥1 + 𝑎32𝑥2 + 𝑎33𝑥3 + ⋯ + 𝑎3𝑛𝑥𝑛 = 𝑏3
𝑎𝑛1𝑥1 + 𝑎𝑛2𝑥2 + 𝑎𝑛3𝑥3 + ⋯ + 𝑎𝑛𝑛𝑥𝑛 = 𝑏𝑛

An n- number of linear equation with n-
number of unknowns is written in
matrix form
𝑎11 ⋯ 𝑎1𝑛
⋮ ⋱ ⋮
𝑎𝑛1 ⋯ 𝑎𝑛𝑛
𝑥1
⋮
𝑥𝑛
=
𝑏1
⋮
𝑏𝑛

 A system of linear
equations can be used in
electrical circuit analysis,
Force analysis in truss,
finite element, finite
difference and also other
areas including non
engineering disciplines as
well.

Another example that requires a solution
of a system of equations is calculating the
force in members of a truss. The forces in
the eight members of the truss shown in
Figure below are determined from the
solution of the following system of eight
equations (equilibrium equations of pins A,
B, C, and D):

0 0.9231 0
1
0
0
0
−1
0
0
0
0
−0.3846
0.9231
−0.3846
0
0
−0.7809
0
−0.7809
0.6247
0
0.6247
0
0 0 0
0
0
0
0
0
−1
1
0
0
0
0
0
0
0
0
0
0.3846
−0.9231
0
0
0
0
0
0.8575
0
0
0
0
−0.5145
0
0
0
0
0
0
0
−1
𝐹𝐴𝐵
𝐹𝐴𝐶
𝐹𝐵𝐶
𝐹𝐵𝐷
𝐹𝐶𝐷
𝐹𝐶𝐸
𝐹𝐷𝐸
𝐹𝐷𝐹
=
1690
0
0
0
0
3625
0
0
It can be written
in Matrix Form

Lower triangular

det(a) = (𝒂𝒊𝒋∗ −𝟏
𝒊 + 𝒋
∗ 𝑴𝒊𝒏𝒐𝒓𝒊𝒋)

 A unique solution (a consistent
set of equations)
 No solution (an inconsistent
set of equations)
 An infinite number of
solutions (a redundant set of
equations)
 The trivial solution, xj =0 (j
=1,2 ..... n), for a homogeneous
set of equations.
𝑓1 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0
𝑓2 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0
𝑓3 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0
𝑓𝑛 𝑥1 + 𝑥2 + 𝑥3 + ⋯ + 𝑥𝑛 = 0

Cramer’s Rule
Consider the system of linear algebraic equations,
Ax= b, which represents n equations. Cramer’s rule
states that the solution for
Where Aj is the n x n matrix obtained by replacing
column j in matrix A by the column vector b.
For example, consider the system of two linear algebraic
equations

Matrix Inverse Method
Systems of linear algebraic equations can be solved using the
matrix inverse, A -1 . Consider the general system of linear
algebraic equations:
Multiplying by
Not all matrices have inverses. Singular matrices, that is, matrices
whose determinant is zero, do not have inverses. The corresponding
system of equations does not have a unique solution.

 Whenever the number of equation in the system of equations become greater
than 3, the methods like Cramer’s, inverse, graphical and substitution becomes
complex and time consuming. So in order to avoid such difficulties numerical
methods are proposed.
 Numerical methods used to solve a system of Linear equations are classified in
to 2.
1. Iterative(Indirect) Methods
1. Gauss Siedel Method
2. Jacobi Method
2. Direct Method
1. Gauss Elimination Method
2. Gauss Jordan Method
3. LU Decomposition Method

Indirect Methods
Initial Approximate solution is assumed and using iterative process a more accurate solution is
obtained successively.
Indirect (Iterative) methods include
1. Gauss Siedel Method
2. Jacobi Method
Direct Methods
A solution is calculated by performing arithmetic operations with the equations. A general form
of equation that is given initially is manipulated to an equivalent system of equations that can
easily be solved. Equivalent system of equations include
1. Lower triangular form of equations
2. Upper triangular form of equations
3. Diagonal form of equations
Direct methods
1. Gauss Elimination (Naïve Gauss)
2. Gauss-Jordan Elimination
3. LU- decomposition

2. Jacobi Method
 An initial (first) value is assumed for each of the unknowns 𝑋1
(1)
, 𝑋2
(1)
, 𝑋3
(1)
, 𝑋4
(1)
, 𝑋5
(1)
…. 𝑋𝑛
(1)
.
 If there is no information regarding the approximate value of the unknowns, the initial values of the
unknowns ca be assumed to be zero.
 The second assumed values of the unknowns 𝑋1
(2)
, 𝑋2
(2)
, 𝑋3
(2)
, 𝑋4
(2)
, 𝑋5
(2)
…. 𝑋𝑛
(2)
are calculated by
substituting the first estimated solution in the right hand side of the equations.
𝑋1
(2)
=
1
𝑎𝑖𝑖
𝑏𝑖 −
𝑗=1,𝑗≠𝑖
𝑗=𝑛
𝑎𝑖𝑗𝑋𝑗
(1)
In general, the (k+1) estimated solution is calculated from the estimate of
𝑋𝑖
(𝐾+1)
=
1
𝑎𝑖𝑖
𝑏𝑖 −
𝑗=𝑖+1
𝑗=𝑛
𝑎𝑖𝑗𝑋𝑖
(𝐾)
The iteration continues until the relative error is small.
𝑋𝑖
(𝐾+1)
− 𝑋𝑖
(𝐾)
𝑋𝑖
(𝐾+1)
< 𝜀𝑃
For i = 1, 2, 3, …n.

Direct Methods
Upper Triangular Form
A system of linear equations that is initially given in general form is reduced to equivalent system of equation by
elimination process. For a system of equations with 4 number of equations
𝒂𝟏𝟏𝒙𝟏 + 𝒂𝟏𝟐𝒙𝟐 + 𝒂𝟏𝟑𝒙𝟑 + 𝒂𝟏𝟒𝒙𝟒 = 𝒃𝟏
𝒂𝟐𝟏𝒙𝟏 + 𝒂𝟐𝟐𝒙𝟐 + 𝒂𝟐𝟑𝒙𝟑 + 𝒂𝟐𝟒𝒙𝟒 = 𝒃𝟐
𝒂𝟑𝟏𝒙𝟏 + 𝒂𝟑𝟐𝒙𝟐 + 𝒂𝟑𝟑𝒙𝟑 + 𝒂𝟑𝟒𝒙𝟒 = 𝒃𝟑
𝒂𝟒𝟏𝒙𝟏 + 𝒂𝟒𝟐𝒙𝟐 + 𝒂𝟒𝟑𝒙𝟑 + 𝒂𝟒𝟒𝒙𝟒 = 𝒃𝟒
Reduced to Upper triangular form which has all zero elements below the diagonal.
𝒂𝟏𝟏𝒙𝟏 + 𝒂𝟏𝟐𝒙𝟐 + 𝒂𝟏𝟑𝒙𝟑 + 𝒂𝟏𝟒𝒙𝟒 = 𝒃𝟏
𝟎𝒙𝟏 + 𝒂′
𝟐𝟐𝒙𝟐 + 𝒂′
𝟐𝟑𝒙𝟑 + 𝒂′
𝟐𝟒𝒙𝟒 = 𝒃𝟐
′
𝟎𝒙𝟏 + 𝟎𝒙𝟐 + 𝒂′
𝟑𝟑𝒙𝟑 + 𝒂′
𝟑𝟒𝒙𝟒 = 𝒃𝟑
′′
𝟎𝒙𝟏 + 𝟎𝒙𝟐 + 𝟎𝒙𝟑 + 𝒂′
𝟒𝟒𝒙𝟒 = 𝒃𝟒
′′′
Using back substitution the value of unknowns will be computed
 It begins at the last equation and the value of Xn will be computed.
 Then the value of X4 will be substituted in the next to the last equation , that is solved for the value of Xn-1
 The process continues in the same manner all the way up to equation 1, which is solved for X1 .
𝑋𝑛 =
𝑏𝑛
𝑎𝑛𝑛
𝑋𝑖 = 𝑏𝑖 −
𝑗=𝑖+1
𝑗=𝑛
𝑎𝑖𝑖
For i = n-1, n-2, n-3, . . . 1

Lower Triangular Form
 A system of linear equations that is initially given in general form is reduced to equivalent system of
equation by elimination process. For a system of equations with 4 number of equations
𝒂𝟏𝟏𝒙𝟏 + 𝒂𝟏𝟐𝒙𝟐 + 𝒂𝟏𝟑𝒙𝟑 + 𝒂𝟏𝟒𝒙𝟒 = 𝒃𝟏
𝒂𝟐𝟏𝒙𝟏 + 𝒂𝟐𝟐𝒙𝟐 + 𝒂𝟐𝟑𝒙𝟑 + 𝒂𝟐𝟒𝒙𝟒 = 𝒃𝟐
𝒂𝟑𝟏𝒙𝟏 + 𝒂𝟑𝟐𝒙𝟐 + 𝒂𝟑𝟑𝒙𝟑 + 𝒂𝟑𝟒𝒙𝟒 = 𝒃𝟑
𝒂𝟒𝟏𝒙𝟏 + 𝒂𝟒𝟐𝒙𝟐 + 𝒂𝟒𝟑𝒙𝟑 + 𝒂𝟒𝟒𝒙𝟒 = 𝒃𝟒
Reduced to lower triangular form which has all zero elements above the diagonal.
𝒂𝟏𝟏𝒙𝟏 + 𝟎𝒙𝟐 + 𝟎𝒙𝟑 + 𝟎𝒙𝟒 = 𝒃𝟏
𝒂′
𝟐𝟏𝒙𝟏 + 𝒂′
𝟐𝟐𝒙𝟐 + 𝟎𝒙𝟑 + 𝟎𝒙𝟒 = 𝒃𝟐
′
+𝒂′′
𝟑𝟐𝒙𝟐 + 𝒂′′
𝟑𝟑𝒙𝟑 + 𝟎𝒙𝟒 = 𝒃𝟑
′′
𝒂′′′
𝟒𝟏𝒙𝟏 + 𝒂′′′
𝟒𝟐𝒙𝟐 + 𝒂′′′
𝟒𝟑𝒙𝟑 + 𝒂′
′′𝟒𝟒𝒙𝟒 = 𝒃𝟒
′′′
Using forward substitution the value of unknowns will be computed
 It begins at the first equation and the value of X1 will be computed.
 Then the value of X1 will be substituted in the next to the first equation , that is solved for the value of X2
 The process continues in the same manner all the way down to the last equation, which is solved for Xn .
𝑋1 =
𝑏1
𝑎11
𝑋𝑖 = 𝑏𝑖 −
𝑗=𝑖
𝑗=𝑖−1
𝑎𝑖𝑖
For i = 2, 3, 4…n

Diagonal Form
 A system of linear equations that is initially given in general form is reduced to
equivalent system of equation by elimination process. For a system of equations
with 4 number of equations
𝒂𝟏𝟏𝒙𝟏 + 𝒂𝟏𝟐𝒙𝟐 + 𝒂𝟏𝟑𝒙𝟑 + 𝒂𝟏𝟒𝒙𝟒 = 𝒃𝟏
𝒂𝟐𝟏𝒙𝟏 + 𝒂𝟐𝟐𝒙𝟐 + 𝒂𝟐𝟑𝒙𝟑 + 𝒂𝟐𝟒𝒙𝟒 = 𝒃𝟐
𝒂𝟑𝟏𝒙𝟏 + 𝒂𝟑𝟐𝒙𝟐 + 𝒂𝟑𝟑𝒙𝟑 + 𝒂𝟑𝟒𝒙𝟒 = 𝒃𝟑
𝒂𝟒𝟏𝒙𝟏 + 𝒂𝟒𝟐𝒙𝟐 + 𝒂𝟒𝟑𝒙𝟑 + 𝒂𝟒𝟒𝒙𝟒 = 𝒃𝟒
Reduced to Diagonal form which has all zero elements on the off diagonal with
normalized diagonal.
𝒙𝟏 + 𝟎𝒙𝟐 + 𝟎𝒙𝟑 + 𝟎𝒙𝟒 = 𝒃𝟏
𝟎𝒙𝟏 + 𝒙𝟐 + 𝟎𝒙𝟑 + 𝟎𝒙𝟒 = 𝒃𝟐
′
𝟎𝒙𝟏 + 𝟎𝒙𝟐 + 𝒙𝟑 + 𝟎𝒙𝟒 = 𝒃𝟑
′′
𝟎𝒙𝟏 + 𝟎𝒙𝟐 + 𝟎𝒙𝟑 + 𝒙𝟒 = 𝒃𝟒
′′′
It is the most reduced form and it is easily solvable.
𝑋𝑖 = 𝑏𝑖
For i = 1,2, 3, 4…n

 A system of linear equations that is initially given in general form is reduced to
equivalent system of equation by elimination process. For a system of equations
with 4 number of equations
𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3 + 𝑎14𝑥4 = 𝑏1
𝑎21𝑥1 + 𝑎22𝑥2 + 𝑎23𝑥3 + 𝑎24𝑥4 = 𝑏2
𝑎31𝑥1 + 𝑎32𝑥2 + 𝑎33𝑥3 + 𝑎34𝑥4 = 𝑏3
𝑎41𝑥1 + 𝑎42𝑥2 + 𝑎43𝑥3 + 𝑎44𝑥4 = 𝑏4
Reduced to Diagonal form which has all zero elements on the off diagonal with
normalized diagonal.
𝑥1 + 0𝑥2 + 0𝑥3 + 0𝑥4 = 𝑏1
0𝑥1 + 𝑥2 + 0𝑥3 + 0𝑥4 = 𝑏2
′
0𝑥1 + 0𝑥2 + 𝑥3 + 0𝑥4 = 𝑏3
′′
0𝑥1 + 0𝑥2 + 0𝑥3 + 𝑥4 = 𝑏4
′′′
It is the most reduced form and it is easily solvable.
𝑋𝑖 = 𝑏𝑖
For i = 1,2, 3, 4…n

Gauss Elimination Method
 Is a procedure for solving a system of linear equations of the form
[a][x]=[b].
 A system of equation that is initially given in general form is reduced
in to upper triangular form of system of equations.
 It is solved using backward substitution Method.
 The first equation in the equivalent is the same as the general.
 In the second equation x1 is eliminated.
 In the third equation x1and x2 are eliminated.

Gauss Elimination Procedure
For a system of equations with four equations and four unknowns.
Step 1: The first equation is unchanged and it is used as a pivot equation.
The first coefficient in the pivot equation is called pivot coefficient. a11 – pivot coefficient
The terms that include x1 in all other equations below the pivot row are eliminated.
 To eliminate a21x1 from the general system of equation , the pivot equation is multiplied by m21 =
a21/a11 and then thee equation is subtracted from the equation consisting the eliminated term (the
second equation of the general form).
𝑴𝒐𝒅𝒊𝒇𝒊𝒆𝒅 𝑬𝒒𝒖𝒂𝒕𝒊𝒐𝒏 = Equation with the term to be eliminated − 𝑴𝒖𝒍𝒕𝒊𝒑𝒍𝒊𝒆𝒓 ∗ (𝑷𝒊𝒗𝒐𝒕 𝒆𝒒𝒖𝒂𝒕𝒊𝒐𝒏)
𝑅2
′
= 𝑅2 − 𝑚21𝑅1
𝑅2
′
= 𝑅2 −
𝑎21
𝑎11
𝑅1
Next the term a21x1 will be eliminated from the general system of equation , the pivot equation is
multiplied by m31 = a31/a11 and then thee equation is subtracted from the equation consisting the
eliminated term (the thirs equation of the general form).
𝑅3
′
= 𝑅3 − 𝑚31𝑅1
𝑅3
′
= 𝑅3 −
𝑎31
𝑎11
𝑅1

Next the term a41x1 will be eliminated from the general system of
equation , the pivot equation is multiplied by m41 = a41/a11 and then
the equation is subtracted from the equation consisting the
eliminated term (the fourth equation of the general form).
𝑅4
′
= 𝑅4 − 𝑚41𝑅1
𝑅4
′
= 𝑅4 −
𝑎41
𝑎11
𝑅1
At the end of step 1, the equation becomes
𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3 + 𝑎14𝑥4 = 𝑏1
0𝑥1 + 𝑎′
22𝑥2 + 𝑎′
23𝑥3 + 𝑎′
24𝑥4 = 𝑏2
′
0𝑥1 + 𝑎′
32𝑥2 + 𝑎′
33𝑥3 + 𝑎′
34𝑥4 = 𝑏3
′
0𝑥1 + 𝑎′
42𝑥2 + 0𝑥3 + 𝑎′
44𝑥4 = 𝑏4
′

Step 2: the pivot equation will be updated to the second equation. The first and the
modified second equation will be unchanged.
All terms with X2 will be eliminated from all equations below the pivot equation.
 To eliminate a32x2 from the general system of equation , the pivot equation is
multiplied by m32 = a32/a22 and then the equation is subtracted from the equation
consisting the eliminated term (the third equation of the general form).
𝑅3
′′
= 𝑅3
′
− 𝑚32𝑅2
′
𝑅3
′′
= 𝑅3
′
−
𝑎32
𝑎22
𝑅2
′
 Then to eliminate a42x2 from the general system of equation , the pivot equation
is multiplied by m42 = a42/a22 and then the equation is subtracted from the
equation consisting the eliminated term (the third equation of the general form).
𝑅4
′′
= 𝑅4
′
− 𝑚42𝑅2
′
𝑅4
′′
= 𝑅4
′
−
𝑎42
𝑎22
𝑅2
′

𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3 + 𝑎14𝑥4 = 𝑏1
0𝑥1 + 𝑎′
22𝑥2 + 𝑎′
23𝑥3 + 𝑎′
24𝑥4 = 𝑏2
′
0𝑥1 + 0𝑥2 + 𝑎′′
33𝑥3 + 𝑎′′
34𝑥4 = 𝑏3
′′
0𝑥1 + 0𝑥2 + 𝑎′′
43𝑥3 + 𝑎′′44𝑥4 = 𝑏4
′′
Step 3: the pivot equation will be updated to the third equation. The first, the
modified second and the twice modified third equation will be unchanged.
All terms with X3 will be eliminated from all equations below the pivot equation.
 To eliminate a43x3 from the general system of equation , the pivot equation is
multiplied by m43 = a43/a33 and then the equation is subtracted from the equation
consisting the eliminated term (the fourth equation of the general form).
𝑅4
′′′
= 𝑅4
′′
− 𝑚43𝑅3
′′
𝑅4
′′
= 𝑅4
′′
−
𝑎43
𝑎22
𝑅3
′′

𝑎11𝑥1 + 𝑎12𝑥2 + 𝑎13𝑥3 + 𝑎14𝑥4 = 𝑏1
0𝑥1 + 𝑎′
22𝑥2 + 𝑎′
23𝑥3 + 𝑎′
24𝑥4 = 𝑏2
′
0𝑥1 + 0𝑥2 + 𝑎′′
33𝑥3 + 𝑎′′
34𝑥4 = 𝑏3
′′
0𝑥1 + 0𝑥2 + 0𝑥3 + 𝑎′
′′44𝑥4 = 𝑏4
′′′
Finally the general form is reduced to upper triangular
form of a system of equations. Which is ready to be
solved using backward substitution method.

Solve a system of linear equations with 4-
unknowns.

General Procedure of Gauss Elimination method to a system with n-number of equations
 The elimination procedure starts with the first row as a pivot row and continues row after
row down to one row before the last.
 At each step, pivot row is used to eliminate that are below the pivot element in all the
rows that are below
 Once the original system of equation is changed to upper triangular form, back
substitution is used for determining the values of the unknowns.
Difficulties with Gauss Elimination
1. The pivot element is zero
2. The pivot element is small relative to other terms in the pivot row
Solutions
1. Pivoting
2. Increasing number of significant figures.
Selecting Pivot Row
 Pivot row with non-zero pivot coefficient.
 Pivot coefficient with greater absolute value than the rest of the elements of the row.

Gauss Jordan Elimination Method
 Similar to Naïve-Gauss, it is also a procedure for solving a system of linear equations
of the form [a][x]=[b] with a small modification.
 A system of equation that is initially given in general form is reduced in to diagonal
form of system of equations with normalized diagonal.
𝑥1 + 0𝑥2 + 0𝑥3 + 0𝑥4 = 𝑏1
′′′
0𝑥1 + 𝑥2 + 0𝑥3 + 0𝑥4 = 𝑏2
′′′
0𝑥1 + 0𝑥2 + 𝑥3 + 0𝑥4 = 𝑏3
′′
′
0𝑥1 + 0𝑥2 + 0𝑥3 + 𝑥4 = 𝑏4
′′′
 It is the most easily solvable equivalent system of equations.
𝑋𝑖 = 𝑏𝑖
′′′

Gauss Jordan Elimination Procedure
 The pivot equation is normalized by dividing all the terms in the equation by pivot
coefficient.
 The normalized pivot equation is used to eliminate the off diagonal terms in all the other
equations.
𝑴𝒐𝒅𝒊𝒇𝒊𝒆𝒅 𝑹𝒐𝒘 = Row with the term to be eliminated − (
𝑎𝑖𝑗
1
) ∗ (𝑵𝒐𝒓𝒎𝒂𝒍𝒊𝒛𝒆𝒅 𝑷𝒊𝒗𝒐𝒕 𝒆𝒒𝒖𝒂𝒕𝒊𝒐𝒏)
𝑅𝑖
′′
= 𝑅𝑖
′
− (
𝑎𝑖𝑗
1
)𝑅𝑁𝑃
′

 Both in Naïve Gauss and Gauss Jordan Elimination Methods
 𝑎 𝑋 = 𝑏 is transformed in to 𝑎′
𝑋 = 𝑏′
 If one needs to solve 𝑎 𝑋 = 𝑏 for different values of [b], in both methods the whole
process is repeated since [b] is altered In the process.
 To avoid this unnecessary repetitive computation there are two methods
 Inverse Method
 LU decomposition Method
 But inverse method is also tedious and time consuming.
LU decomposition Method
LU Decomposition is another method to solve a set of simultaneous linear equations.
 For most non-singular matrix [A] that one could conduct Naïve Gauss Elimination
forward elimination steps, one can always write it as:
[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix

When solving a set of linear equations
If [a] = [L][U] then [a][x] = [b]
Which gives
[L][U][x] = [b]
Which is not easily solvable,
Let [U][x] = [y]…(1)
Substituting back in to the equation
[L][y]=[b]….(2)
Then solve equation 2 for y using forward substitution.
Then solve equation 1 for x using backward substitution.
There are 2 method in which [L] and [U] are determined
1. LU decomposition Using Gauss Elimination Method
2. LU decomposition Using Crout’s Method

LU decomposition Using Gauss Elimination Method
 When the Gauss-Elimination method is applied to matrix
[a], the elements of [U] and [L] are computed.
 [U] is the matrix coefficient [a] that is obtained at the end of
the procedure.
 [L] is not written explicitly in the procedure, but the
elements that make up the matrix are calculated along the
way.
 The elements of [L] on the diagonal are all 1’s and
 The elements below the diagonal are the multiplier mij that
multiply the pivot equation in gauss elimination procedure.
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
=
1
𝑚21
𝑚31
𝑚41
0
1
𝑚32
𝑚42
0
0
1
𝑚43
0
0
0
1
𝑎11
0
0
0
𝑎12
𝑎22
′
0
0
𝑎13
𝑎23
′
𝑎33
′′
0
𝑎14
𝑎24
′
𝑎34
′′
𝑎44
′′′

SolvingSystemof LinearEquations

LU decomposition Using Crout’s Method
 In this method matrix [a] is decomposed in to the product
[L][U}, where the diagonal of [U] are all 1’s.
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
=
𝐿11
𝐿21
𝐿31
𝐿41
0
𝐿22
𝐿32
𝐿42
0
0
𝐿33
𝐿43
0
0
0
𝐿44
1
0
0
0
𝑈12
1
0
0
𝑈13
𝑈23
1
0
𝑈14
𝑈24
𝑈34
1
 Executing matrix multiplication on the right hand side.
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
=
𝐿11
𝐿21
𝐿31
𝐿41
𝐿11𝑈12
𝐿11𝑈12 + 𝐿22
𝐿31𝑈12 + 𝐿32
𝐿41𝑈12 + 𝐿42
𝐿11𝑈13
𝐿11𝑈13 + 𝐿22𝑈23
𝐿31𝑈13 + 𝐿33
𝐿41𝑈13 + 𝐿42𝑈23 + 𝐿43
𝐿11𝑈14
𝐿11𝑈14 + 𝐿22𝑈24
𝐿31𝑈13 + 𝐿32𝑈23 + 𝐿33
𝐿41𝑈14 + 𝐿42𝑈24+𝐿43𝑈34 + 𝐿44

𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
=
𝐿11
𝐿21
𝐿31
𝐿41
𝐿11𝑈12
𝐿11𝑈12 + 𝐿22
𝐿31𝑈12 + 𝐿32
𝐿41𝑈12 + 𝐿42
𝐿11𝑈13
𝐿11𝑈13 + 𝐿22𝑈23
𝐿31𝑈13 + 𝐿33
𝐿41𝑈13 + 𝐿42𝑈23 + 𝐿43
𝐿11𝑈14
𝐿11𝑈14 + 𝐿22𝑈24
𝐿31𝑈13 + 𝐿32𝑈23 + 𝐿33
𝐿41𝑈14 + 𝐿42𝑈24+𝐿43𝑈34 + 𝐿44
 The elements of [L] and [U] are determined by equation the corresponding elements of the matrices on both
sides of the equation above.
 From the first column
𝐿11 = 𝑎11
𝐿21 = 𝑎21
𝐿31 = 𝑎31
𝐿41 = 𝑎41
 Moving to the first row, 𝑈12 =
𝑎12
𝐿11
, 𝑈13 =
𝑎13
𝐿11
, 𝑈14 =
𝑎14
𝐿11
 Moving to the second row, 𝐿22 = 𝑎22 − 𝐿21𝑈12, 𝑈23 =
𝑎23−𝐿21𝑈13
𝐿22
, 𝑈24 =
𝑎24−𝐿21𝑈14
𝐿22
 The process continues in similar manner to the third and fourth row.

𝐼𝑓 𝑚𝑎𝑡𝑟𝑖𝑥 𝑎 𝑖𝑠 (𝑛𝑥𝑛) 𝑚𝑎𝑡𝑟𝑖𝑥, 𝑡ℎ𝑒 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑜𝑓 𝐿 𝑎𝑛𝑑 𝑈 𝑎𝑟𝑒 𝑔𝑖𝑣𝑒𝑛 𝑏𝑦
Step 1: Calculating the first column of [L]
For i = 1,2,3,…n 𝐿𝑖 = 𝑎𝑖1
Step 2: Substituting 1’s in the diagonal of [U]
For i = 1,2,3,…n 𝑈𝑖𝑖 = 1
Step 3: Calculating the elements of [U] in the first row except 𝑈11 , 𝑤ℎ𝑖𝑐ℎ 𝑖𝑠 𝑎𝑙𝑙𝑟𝑒𝑎𝑑𝑦 𝑘𝑛𝑜𝑤𝑛.
For j = 2,3,4, …n 𝑈𝑖𝑗 =
𝑎1𝑗
𝐿11
Step 4: Calculating the rest of the elements row after row.
 The elements of [L] are calculated first and are used to calculate the elements of [U].
For i = 2,3,4, …n
For j = 2,3,4, …i
𝐿𝑖𝑗 = 𝑎𝑖𝑗 −
𝐾=1
𝐾=𝑗−1
𝐿𝑖𝐾𝑈𝐾𝑗
𝑈𝑖𝑗 =
𝑎𝑖𝑗 − 𝐾=1
𝐾=𝑗−1
𝐿𝑖𝐾𝑈𝐾𝑗
𝐿𝑖𝑖

LU decomposition with Pivoting
 For a successful execution of the gauss-elimination, pivoting may be required.
 Sometimes there may be a need of pivoting in Crout’s decomposition method as well.
 Therefore pivoting is employed in the elimination process,
[a]≠ [L][U]
 Instead
[P][a] = [L][U]
 Where [P] is called permutation matrix,
is composed of 1’s and 0’s
is used to trace the change made in the order of the rows.
[P][a][x]=[P][b]=[L][U][x]

Iterative Methods
 the equations are written in explicit form
 Each unknown is written in terms of other unknowns.
 For a system of four equations
 The solution process starts by assuming initial value for the unknowns (first estimated solution).
 In the first iteration the first estimated solution is substituted in the right hand side equations
 The new values( the second estimated solution) are calculated for the unknowns.
 In the second iteration the second estimated solution is substituted in the right hand side equations
 The new values( the third approximated solution) are calculated for the unknowns.
 The process continues in the same manner, until it converged to the exact values or reaches predetermined
error tolerance
 For a system with n-equations , the explicit equation for Xi unknowns are:-
𝑋𝑖 =
1
𝑎𝑖𝑖
𝑏𝑖 −
𝑗=1,𝑗≠𝑖
𝑗=𝑛
For i=1,2,3,..n

Condition of Convergence
 For a system of n-equations of the form [a][X]=[b] a sufficient but not necessary condition is
𝑎𝑖𝑖 >
𝑗=1,𝑗≠𝑖
𝑗=𝑛
𝑎𝑖𝑗
 The absolute value of the diagonal element is greater than the sum of the absolute value of the off-
diagonal elements.
 If satisfied, [a] is diagonally dominant.
1. Gauss-Siedel Method
 An initial (first) value is assumed for the unknowns 𝑋2
(1)
, 𝑋3
(1)
, 𝑋4
(1)
, 𝑋5
(1)
…. 𝑋𝑛
(1)
except 𝑋1
(1)
.
 If there is no information regarding the approximate value of the unknowns, the initial values of the
unknowns ca be assumed to be zero.
 The first assumed values of the unknowns are substituted in the equation with i=1, to calculate for
X1 .
 Next with i = 2 X2 --------, i = n  Xn then end of first iteration.
 In Gauss-Siedel, the current values of the unknowns are used to calculate the next values of the next
unknown.
 In Jacobi Method, the values of the unknowns obtained in one iteration are used as a complete set, in
the next iteration.

𝑋1
(𝐾+1)
=
1
𝑎11
𝑏1 −
𝑗=2
𝑗=𝑛
𝑎1𝑗𝑋𝑗
(𝐾)
𝑋𝑖
(𝐾+1)
=
1
𝑎𝑖𝑖
𝑏𝑖 −
𝑗=1
𝑗=𝑖−1
(𝐾+1)
+
𝑗=𝑖+1
𝑗=𝑛
(𝐾)
𝑋𝑛
(𝐾+1)
=
1
𝑎𝑛𝑛
𝑏𝑛 −
𝑗=1
𝑗=𝑛−1
𝑎1𝑗𝑋𝑗
(𝐾+1)
Solve using Gauss Siedel Method
𝟖𝒙𝟏 + 𝟐𝒙𝟐 + 𝟑𝒙𝟑 = 𝟓𝟏
𝟐𝒙𝟏 + 𝟓𝒙𝟐 + 𝒙𝟑 = 𝟐𝟑
−𝟑𝒙𝟏 + 𝒙𝟐 + 𝟔𝒙𝟑 = 𝟐𝟎
Take zero as first (initial) estimated values of the unknowns.

Inverse of A matrix
 The inverse of square matric [a] is the matrix [a]-1
 The product of a matrix with its inverse gives Identity matrix [I].
[a] [a]-1 = [a]-1[a] =[I]
 The process of calculating the inverse of a matrix is essentially the same as the process of solving a system of
linear equations.
 For a matrix of (4x4)  if [a] is the given matrix and [x] is the inverse of [a], which is unknow, then
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
𝑥11
𝑥21
𝑥31
𝑥41
𝑥12
𝑥22
𝑥32
𝑥42
𝑥13
𝑥23
𝑥33
𝑥43
𝑥14
𝑥24
𝑥34
𝑥44
=
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1

 Can be written as a four separate system of equations
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
𝑥11
𝑥21
𝑥31
𝑥41
=
1
0
0
0
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
𝑥12
𝑥22
𝑥32
𝑥42
=
0
1
0
0
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
𝑥12
𝑥22
𝑥32
𝑥42
=
0
0
1
0
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
𝑥14
𝑥24
𝑥34
𝑥44
=
0
0
0
1
Can be solved using any of the methods we have seen so far.

Inverse of A matrix Using Gauss Jordan Method
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
𝑥11
𝑥21
𝑥31
𝑥41
𝑥12
𝑥22
𝑥32
𝑥42
𝑥13
𝑥23
𝑥33
𝑥43
𝑥14
𝑥24
𝑥34
𝑥44
=
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
The augmented matrix
𝑎11
𝑎21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
𝑎34
𝑎44
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
=
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
𝑏11
′
𝑏21
′
𝑏31
′
𝑏41
′
𝑏12
′
𝑏22
′
𝑏32
′
𝑏42
′
𝑏13
′
𝑏23
′
𝑏33
′′
𝑏43
′
𝑏14
′
𝑏24
′
𝑏34
′
𝑏44
′
Since a matrix is reduced to diagonal matrix, the values of the unknowns are found at the corresponding element
of the constant matrix.
𝑎 −1
= 𝑥 = 𝑏′
For i = 1,2…4 and j = 1,2,..4
𝑥𝑖𝑗 = 𝑏′
𝑖𝑗

Error and Residual
 A numerical solution of a system of equations is seldom exact solution.
 Even though direct method can produce exact solution, even then they are susceptible to round-off errors.
 Solutions that are obtained with iterative method are approximate by nature.
 If [XNS] is a computed approximate solution of a system of equations of the form [a][X]=[b]
 [XTS] is true(exact) solution #
 The true error is a vector
[e] = [XTS] - [XNS] , but cannot in general be computes since [XTS] not known.
 Residual – measures how well the system of equations is satisfied when [XNS] is substituted for [X]. It is
proportional to tolerance in function.
[r] = [a] [XTS] - [a] [XNS]
[r] = [b] - [a] [XNS]
 The vector [r] has n-number of elements and is an alternative measure of the accuracy.
 If the numerical solution is close to the true solution, then all the elements of [r] are small.
 [r] does not really indicate how small the error is. It is possible to have an approximate numerical solution
that has large true error but give small residual error.

Condition Number
The number ||A||||A−1|| is called condition number
Denoted by C(A) = ||A||||A−1||
 Condition number of a system is a measure of the sensitivity of the system to small changes in
any of its elements.
 Ill conditioning is quantified by the condition number of a matrix.
 Which is defined in terms of the norms of matric and its inverse.
 If the system is ill-conditioned and there happens to be round off error which produce a slight
(small) change, the system will respond with large change .
 The condition number of the identity matrix is 1.
 The condition number of any other matrix is 1 or greater.
 If the condition number is approximately 1, then the true relative error is of the same order
of magnitude as the relative residual.
 If the condition number is much larger than 1, then a small relative residual does not
necessarily imply a small true relative error.
 For a given matrix, the value of the condition number depends on the matrix norm that is
used.
 The inverse of a matrix has to be known in order to calculate the condition number of the
matrix.
 Ill-conditioning can be corrected
 Pivoting
 Iterative method

 A more accurate estimate of the error in numerical solution can be obtained by using quantities that measure
the size or magnitude of vectors and matrices.
 Size and magnitude of vectors and matrices are computed by a quantity called NORM.
NORM
 A measure of the magnitude if A, X, and b and it is denoted by ||A||, ||X||, ||b||
Norms have the following properties
 ||A|| > 0
 ||A|| = 0 if and only if A=0
 ||kA||= |k|*||A||
 ||A+B||<=||A|| + ||B||
 ||AB||<= ||A||* ||B||

Norm of a Matrix
Maximum Column Sum
𝐴 1 = 𝑀𝑎𝑥1≤𝑗≤𝑛
𝑖=1
𝑛
𝑎𝑖𝑗
 Maximum Row Sum
𝐴 ∞ = 𝑀𝑎𝑥1≤𝑖≤𝑛
𝑗=1
𝑛
𝑎𝑖𝑗
 Euclidean Norm
𝑋 𝑒 =
𝑖=1
𝑛
𝑗=1
𝑛
𝑎𝑖𝑗
2
1
2
Norm of a Vector
 Sum of Magnitude
𝑋 1 = 𝑋𝑖
 Euclidean Norm
𝑋 2 = 𝑋 𝑒 = 𝑋𝑖
2
1
2
 Maximum Magnitude norm
𝑋 ∞ = 𝑀𝑎𝑥1<𝑖<𝑛
𝑋𝑖

Consider a system of Linear Algebraic Equations
AX = b ------------------------------------(1)
 Applying property of norm
||b||= ||AX|| ≤ ||A||* ||X|| ------------------------------------(2)
 Consider a slight modification of (1) in which b is altered by Δb, which causes a change in the solution ΔX.
A(X + ΔX) = b +Δb -----------------------------------(3)
 Subtracting (1) form (3)
AΔX = Δb -----------------------------------(4)
 Solving (4) for ΔX
ΔX = A-1Δb -----------------------------------(5)
 Applying norm property on (5)
||ΔX ||= ||A-1Δb|| ≤ ||A-1||*||Δb|| -----------------------------------(6)
 Multiplying (2) and (6)
||b||* ||ΔX || ≤ ||A||* ||X|| *||A-1||*||Δb|| -----------------------------------(7)
 Diving (7) by ||b||* ||X ||
||ΔX ||
||X ||
≤ ||A||||A−1||
||Δb||
||b||
Where C(A)=||A||||A−1|| it can also be computed for ΔA
||ΔX ||
||X ||
≤ C(A)
||ΔA||
||A||

Computational Method for Engineers: Solving a system of Linear Equations

Recommended

Recommended

More Related Content

Similar to Computational Method for Engineers: Solving a system of Linear Equations

Similar to Computational Method for Engineers: Solving a system of Linear Equations (20)

Recently uploaded

Recently uploaded (20)

Computational Method for Engineers: Solving a system of Linear Equations