1. Numerical solution of ordinary and
partial dierential Equations
Module 19: Newton's method for system of
equations
Dr.rer.nat. Narni Nageswara Rao£
1 Newton's method for system of equations
The Newton's method (in the one-dimensional case) as a functional iteration
scheme xn = g(xn 1), for n ! 1 is
g(x) = x (x)f(x)
where (x) = 1
2f0
(x)
, assuming that fH(x) T= 0 gives a quadratic convergence
(see B.Tech 1st year).
Using a similar approach in the n-dimensional case involves a matrix
A(x) =
2
6
6
4
a11(x) a12(x) ¡¡¡ a1n(x)
a21(x) a22(x) ¡¡¡ a2n(x)
¡¡¡ ¡¡¡ ¡¡¡ ¡¡¡
an1(x) an2(x) ¡¡¡ ann(x)
3
7
7
5
(1)
where each of the entries aij(x) is a function from Rn
into R. This requires
that A(x) be found so that
G(x) = x A(x) 1
F(x)
gives quadratic convergence to the solution of F(x) = 0, assuming that A(x)
is non singular at the
3. Theorem 1.1. Let p be a solution of G(x) = x. Suppose a number 0
exists with
(i) @gi
@xj
is continuous on N = fx=kx pk g; for each i = 1;2;¡¡¡ ;n
and j = 1;2;¡¡¡ ;n
(ii) @2gi(x)
@xj @xk
is continuous, and
9. M for some constant M, where
x P N, for each i = 1;2;¡¡¡ ;n;j = 1;2;¡¡¡ ;n and k = 1;2;¡¡¡ ;n
(iii) @gi(p)
@xk
= 0; for each i = 1;2;¡¡¡ ;n and k = 1;2;¡¡¡ ;n.
Then a number ^ exists such that the sequence generated by x
(k)
=
G(x
(k 1)
) converges quadratically to p for any choice of x
(0)
, provided that
kx
(0)
pk ^, moreover,
kx
(k)
pkI
n2
M
2
kx
(k 1)
pk2
I; for each k ! 1:
Suppose A(x) is an n¢n matrix of functions from Rn
into R in the from
of (1). Assume that A(x) is non-singular near a solution p of F(x) = 0, and
let bij(x) denote the entry of A(x) 1
in the ith
row and jth
column.
Since G(x) = X A(x) 1F(x), we have
gi(x) = xi
nX
j=1
bij(x)fj(x)
and
@gi
@xk
(x) =
8
:
1 Pn
j=1
bij(x)@fj
@xk
(x) + @bij
@xk
(x)fj(x)
; if i = k
Pn
j=1
bij(x)@fj
@xk
(x) + @bij
@xk
(x)fj(x)
; if i T= k
From Theorem 1.1 implies that we need @gi(p)
@xk
= 0, for each i = 1;2;¡¡¡ ;n
and k = 1;2;¡¡¡ ;n. This means that for i = k,
0 = 1
nX
j=1
bij(p)
@fj
@xi
(p)
so nX
j=1
bij(p)
@fj
@xi
(p) = 1 (2)
2
10. when k T= i,
0 =
nX
j=1
bij(p)
@fj
@xi
(p)
so nX
j=1
bij(p)
@fj
@xi
(p) = 0 (3)
De
11. ning the matrix J(x) by
J(x) =
2
6
6
6
4
@f1
@x1
(x) @f1
@x2
(x) ¡¡¡ @f1
@xn
(x)
@f2
@x1
(x) @f2
@x2
(x) ¡¡¡ @f2
@xn
(x)
...
...
...
@fn
@x1
(x) @fn
@x2
(x) ¡¡¡ @fn
@xn
(x)
3
7
7
7
5
(4)
From the conditions (2) and (3)
A(p) 1
J(p) = I
the identity matrix.
so
A(p) = J(p)
An appropriate choice for A(x) is, consequently, A(x) = J(x) since this sat-
is
13. ned by
G(x) = x J(x) 1
F(x)
and the functional iteration procedure evolves from selecting x
(0)
and gener-
ating, for k ! 1
x
(k)
= G(x
(k 1)
) = x
(k 1)
J(x
(k 1)
)F(x
(k 1)
) (5)
This is called Newton's method for nonlinear systems, and it is gen-
erally expected to give quadratic convergence, provided that a suciently
accurate starting value is known and J(p) 1
exists.
The matrix J(x) is called the Jacobian matrix.
3
14. The weakness in Newton's method arises from the need to compute and
invert the matrix J(x) at each step.
Technique: In practice, explicit computation of J(x) 1
is avoided by
performing the opertion in a two-step manner. First a vector y is found
that satis
15. es J(x)(k 1)
y = F(x
(k 1)
). Then the new approximation, x
(k)
is
obtained by adding y to x
(k 1)
.
Algorithm with two step procedure
AIM: To approximate the solution of the nonlinear system F(x) = 0
given an initial approximation.
INPUT: number n of equations and unknowns initial approximation
x = (x1;x2;¡¡¡ ;xn), tolerance TOL; maximum number of iterations N.
OUTPUT: approximate solution x = (x1;x2;¡¡¡ ;xn)T
or
a message that the number of iterations was exceeded.
Step 1: Set k = 1
Step 2: While (k N) do steps 3 7.
Step 3: Calculate F(x) and J(x), where
J(x)i;j =
@fi(x)
@xj
for 1 i;j n
Step 4: Solve the n ¢n linear system J(x)y = F(x).
Step 5: Set x = x + y
Step 6: If kyk TOL then OUTPUT(x)
(the procedure was successful.)
STOP
Step 7: Set k = k + 1.
Step 8: OUTPUT('Maximum number of iterations exceeded');
(The procudure was unsuccessful)
4
16. STOP.
Example Calculate the solution of the system of equations
x3
+ y3
= 53
2y3
+ z4
= 69
3x5
+ 10z2
= 770
which is close to x = 3;y = 3;z = 2:
Solution Take the initial approximation as x0 = 3;y0 = 3;z0 = 2. We
have
) J(x;y;z) =
2
4
3x2
3y2
0
0 6y2
4z3
15x4
0 20z
3
5
) J0 = J(3;3;2) =
2
4
27 27 0
0 54 32
1215 0 40
3
5
and
F0 =
2
4
1
1
1
3
5
for k = 1: The linear system
J0b(0)
= F0
is solved for b(0)
. Then we calculated the solution as
2
4
x(1)
y(1)
z(1)
3
5 =
2
4
x(0)
y(0)
z(0)
3
5 +
2
6
4
b(0)
1
b(0)
2
b(0)
3
3
7
5
=
2
4
2:998
2:9692
2:0319
3
5
k = 2;3; and so on.
Example: Solve the nonlinear system
3x1 cos(x2x3) 1
2
= 0
5
17. x2
1 81(x2 + 0:1)2
+ sinx3 + 1:06 = 0
e x1x2
+ 20x3 +
10 3
3
= 0
using Newton's method with an initial approximation of x
(0)
= (0:1;0:1; 0:1)T
.
Solution:
Let f1(x1;x2;x3) = 3x1 cos(x2x3) 1
2
f2(x1;x2;x3) = x2
1 81(x2 + 0:1)2
+ sinx3 + 1:06
f3(x1;x2;x3) = e x1x2
+ 20x3 +
10 3
3
) J(x1;x2;x3) =
2
4
3 x3 sin(x2x3) x2 sin(x2x3)
2x1 162(x2 + 0:1) cosx3
x2e x1x2 x1e x1x2 20
3
5
and 2
6
4
x(k)
1
x(k)
2
x(k)
3
3
7
5 =
2
6
4
x(k 1)
1
x(k 1)
2
x(k 1)
3
3
7
5 +
2
6
4
y(k 1)
1
y(k 1)
2
y(k 1)
3
3
7
5
at the kth
step, the linear system
J(x(k 1)
)y(k 1)
= F(x(k 1)
)
must be solved. The results are tabulated in the Table 1 for k = 0;1;¡¡¡5:
k x(k)
1 x(k)
2 x(k)
3 kx
(k)
x
(k 1)
kI
0 0:100000000 0:100000000 0:100000000
1 0:50003702 0:01946686 0:52152047 0:422
2 0:50004593 0:00158859 0:52355711 1:79 ¢10 2
3 0:50000034 0:00001244 0:52359845 1:58 ¢10 3
4 0:50000000 0:00000000 0:52359877 1:24 ¢10 5
5 0:50000000 0:00000000 0:52359877 0
Table 1: Solution of the system by Newton's method
6
18. 2 Shooting Methods
2.1 Introduction
Consider the Boundary value problem
yHH = f(t;y;yH); t P (a;b) (6)
with one of the following three boundary conditions.
(i) Boundary conditions of
19. rst kind:
y(a) =
1; y(b) =
2 (7)
(ii) Boundary conditions of second kind:
yH(a) =
1; yH(b) =
2 (8)
(iii) Boundary conditions of third kind (or mixed type):
a0y(a) a1yH(a) =
1; b0y(b) + b1yH(b) =
2 (9)
where a0, b0, a1, b1,
1 and
2 are constants such that
a0a1 ! 0; ja0j+ ja1j T= 0
b0b1 ! 0; jb0j+ jb1j T= 0; ja0j+ jb0j T= 0
2.2 Shooting method(Initial value problem method)
Consider the boundary value problem (6) (BVP) subject to the given bound-
ary conditions.
Since the dierential equation is of second order, we require two linearly
independent conditions to solve the boundary value problems. One of the
ways of solving the boundary value problem is the following:
(i) Boundary conditions of the
20. rst kind. Here, we are given y(a) =
1. In
order that an initial value method can be used, we guess the value of
the slope at t = a as yH(a) = .
(ii) Boundary conditions of the second kind. Here, we are given yH(a) =
1.
In order that an initial value method can be used, we guess the value
of y(t) at t = a as y(a) = .
7
21. (iii) Boundary conditions of third kind. Here we guess the value of y(a) or
yH(a). If we assume that yH(a) = , then from (4), we get
y(a) =
(
1 + a1)
a0
The realted initial value problem is solved upto t = b, by using a single step or
multistep method. If the problem is solved directly, then we use the methods
for second order initial value problems. If the dierential equation is reduced
to a system of two frist order equations. Then we use the Rutnge-Kutta
method or multistep methods for a system of
22. rst order equations.
If the solution at t = b does not satisfy the given boundary condition
at the other end t = b, then we take another guess value of y(a) or yH(a)
and solve the initial value problem again upto t = b. These two solutions at
t = b, of the initial value problems are used to obtain a better estimate of
y(a) or yH(a). A sequence of such problems are solved if necessary to obtain
the solution of the given boundary value problem.
For a linear, non homogeneous boundary value problem, it is sucient
to solve two initial value problems with two linearly independent guess initial
conditions.
This technique of solving the boundary value problems by using the meth-
ods for solving the initial value problems is called the Shooting Method.
2.3 Linear shooting method
Theorem 2.1. Suppose the function f in the boundary value problem
yHH = f(t;y;yH); a t b;y(a) =
1;y(b) =
2
is continuous on the set D = f(t;y;yH)ja t b; I y I; I yH
Ig, and that the partial derivatives fy and fy0 are also continuous on D. If
(i) fy(t;y;yH) 0, for all (t;y;yH) P D, and
(ii) a constant M exists, with
jfy0 (t;y;y)j M; V(x;y;yH) P D
then the boundary value problem has a unique solution.
8
23. Corollary 2.2. If the linear boundary value problem
yHH = p(t)yH + q(t)y + r(t);a t b;y(a) =
1;y(b) =
2
satis
24. es
(i) p(t);q(t) and r(t) are continuous on [a;b],
(ii) q(t) 0 on [a;b] then the problem has a unique solution.
Let us
25. rst consider the initial value problems
yHH = p(t)yH + q(t)y + r(t); a t b;y(a) =
1;yH(a) = 0 (10)
and
yHH = p(t)yH + q(t)y;a t b;y(a) = 0;yH(a) = 1 (11)
From the above corollary 2.2 and from the theory of homogeneous dierential
equations, the problems (10) and (11) have a unique solution. Let y1(t)
denote the solution of (10), y2(t) denote the solution of (11) and let
y(t) = y1(t) +
2 y1(b)
y2(b)
:y2(x) (12)
Then
yH(t) = yH
1(t) +
2 y1(b)
y2(b)
yH
2(t)
and
yHH(t) = yHH
1 (t) +
2 y1(b)
y2(b)
yHH
2 (t)
so
yHH = p(t)yH
1 + q(t)y1 + r(t) +
2 y1(b)
y2(b)
[p(t)yH
2(t) + q(t)y2]
= p(t)
yH
1(t) +
2 y1(b)
y2(b)
yH
2(t)
+ q(t)
y1 +
2 y1(b)
y2(b)
+ r(t)
yHH = p(t)yH(t) + q(t)y(t) + r(t)
Moreover
y(a) = y1(a) +
2 y1(b)
y2(b)
y2(a)
=
1 +
2 y1(b)
y2(b)
:0
=
1
9
26. and
y(b) = y1(b) +
2 y1(b)
y2(b)
:y2(b)
= y1(b) +
2 y1(b) =
2
Hence, y(t) is the unique solution to the linear boundary value problem, pro-
vided, of course, that y2(b) T= 0.
The shooting method for linear equations is based on the replacement of
the linear BVP by the two initial value problems (10) and (11). Numerous
methods are available from the previous modules (module 5, 6, 7, 8, 9, 10,
13, 14, 15, 16) for approximating the solutions y1(t) and y2(t). Once these
approximations are available, the solution to the BVP is approximated using
(12). Graphically the method had the appearence shown in Figure 1.
y
1
a b t
y2(t)
y1(t)
2
y(t) = y1(t) +
2 y1(b)
y2(b)
y2(t)
Figure 1: Linear shooting method.
10
