2. Introduction
Numerical Methods and Application of the Methods
Numerical Methods for solving Differential/Partial differential
equation
Conclusion
References
3. Babylonian clay
tablet YBC 7289 (c.
1800–1600 BC) with
annotations. The
approximation of
the square root of 2 is
four sexagesimal figure
s, which is about
six decimal figures. 1 +
24/60 + 51/602 +
10/603=
1.41421296...[1]
Numerical analysis is the study of algorithms that use numerical approximation. One of the
earliest mathematical writings is a Babylonian tablet from the Yale Babylonian
Collection (YBC 7289), which gives a sexagesimal numerical approximation of the square
root of 2, the length of the diagonal in a unit square. Being able to compute the sides of a
triangle is extremely important, for instance, in astronomy, carpentry and construction.
Numerical analysis continues this long tradition of practical mathematical calculations.
Numerical analysis naturally finds applications in all fields of engineering and the physical
sciences, but in the 21st century also the life sciences and even the arts have adopted
elements of scientific computations. Ordinary differential equations appear in celestial
mechanics (planets, stars and galaxies); numerical linear algebra is important for data
analysis; stochastic differential equations and Marko chains are essential in simulating living
cells for medicine and biology. Before the advent of modern computers numerical
methods often depended on hand interpolation in large printed tables. Since the mid 20th
century, computers calculate the required functions instead. These same interpolation
formulas nevertheless continue to be used as part of the software algorithms for
solving differential equations.
4. What is Interpolation ?
Let given (x0,y0), (x1,y1), …… (xn,yn), find the value of ‘y’ at a value of ‘x’ that is not given
5. Polynomials are the most common choice of interpolants because they are easy to:
Evaluate
Differentiate, and
Integrate
Lagrangian Interpolation :-
This is again an Nth degree polynomial approximation formula to the function f(x), which is known at discrete
points xi, i = 0, 1, 2 . . . Nth. The formula can be derived from the Vandermonds determinant but a much
simpler way of deriving this is from Newton's divided difference formula. If f(x) is approximated with an Nth
degree polynomial then the Nth divided difference of f(x) constant and (N+1)th divided difference is zero. That is
f [x0, x1, . . . xn, x] = 0
From the second property of divided difference we can write
f0
(x0 − x1) . . . (x0 − xn)(x0 − x)
+
fn
(xn − x0) . . . (xn − xn−1)(xn − x)
+…….+
fx
(x − x0) . . . (x − xn)
= 0
6. Or
f(x) =
(x − x1) . . . (x − xn)
(x0 − x1) . . . (x0 − xn)
f0 +…….+
(x − x0) . . . (x − xn−1)
(xn − x0) . . . (xn − xn−1)
fn
Since Lagrange's interpolation is also an Nth degree polynomial approximation to f(x) and the Nth degree
polynomial passing through (N+1) points is unique hence the Lagrange's and Newton's divided difference
approximations are one and the same. However, Lagrange's formula is more convinent to use in computer
programming and Newton's divided difference formula is more suited for hand calculations.
7. Application of Lagrangian Interpolation formula :-
This formula use to finding the velocity of Rocket
For example:-
The upward velocity of a rocket is given as a function of time in Table. Find the velocity at t=16 seconds
using the Lagrangian method for linear interpolation ?
Table Velocity as a function of time
T(s) :- 0 10 15 20 22.5 30
V(t) (m/s) :- 0 227.04 362.78 517.35 602.97 901.67
9. 10 12 14 16 18 20 22 24
350
400
450
500
550
517.35
362.78
y s
f range
( )
f x desired
x s
1
10
x s
0
10
x s range
x desired
𝑣 𝑡 =
𝑖=0
1
𝐿𝑖 𝑡 𝑣(𝑡𝑖)
12. Numerical methods in fluid mechanics :-
Fluid motion is governed by the Navier–Stokes equations, a set of coupled and nonlinear partial
differential equations derived from the basic laws of conservation of mass, momentum and energy. The
unknowns are usually the flow velocity, the pressure and density and temperature. The analytical solution of
this equation is impossible hence scientists resort to laboratory experiments in such situations. The answers
delivered are, however, usually qualitatively different since dynamical and geometric similitude are difficult to
enforce simultaneously between the lab experiment and the prototype. Furthermore, the design and
construction of these experiments can be difficult (and costly), particularly for stratified rotating flows.
Computational fluid dynamics (CFD) is an additional tool in the arsenal of scientists. In its early days CFD
was often controversial, as it involved additional approximation to the governing equations and raised
additional (legitimate) issues. Nowadays CFD is an established discipline alongside theoretical and
experimental methods. This position is in large part due to the exponential growth of computer power which
has allowed us to tackle ever larger and more complex problems.
13. Discretization :-
The central process in CFD is the process of discretization, i.e. the process of taking differential equations
with an infinite number of degrees of freedom, and reducing it to a system of finite degrees of freedom.
Hence, instead of determining the solution everywhere and for all times, we will be satisfied with its
calculation at a finite number of locations and at specified time intervals. The partial differential equations are
then reduced to a system of algebraic equations that can be solved on a computer. Errors creep in during the
discretization process. The nature and characteristics of the errors must be controlled in order to ensure that:
1. we are solving the correct equations (consistency property)
2. that the error can be decreased as we increase the number of degrees of freedom (stability
and convergence).
Once these two criteria are established, the power of computing machines can be leveraged to solve
the problem in a numerically reliable fashion. Various discretization schemes have been developed to cope
with a variety of issues. The most notable for our purposes are: finite difference methods, finite volume
methods, finite element methods, and spectral methods.
14. Finite difference method :-
Finite difference replace the infinitesimal limiting process of derivative calculation:
lim
∆𝑥→0
𝑓′
𝑥 =
𝑓 𝑥 + ∆𝑥 − 𝑓 𝑥
∆𝑥
with a finite limiting process,i.e.
𝑓′
𝑥 =
𝑓 𝑥 + ∆𝑥 − 𝑓 𝑥
∆𝑥
+𝑂 ∆𝑥
The term 𝑂 ∆𝑥 gives an indication of the magnitude of the error as a function of the mesh spacing.
15. In this instance, the error is halfed if the grid spacing, x is halved, and we say that this is a first order
method. Most FDM used in practice are at least second order accurate except in very special circumstances.
Finite Difference method is still the most popular numerical method for solution of PDEs because of their
simplicity, efficiency and low computational cost. Their major drawback is in their geometric inflexibility which
complicates their applications to general complex domains. These can be alleviated by the use of either
mapping techniques and/or masking to fit the computational mesh to the computational domain.
Finite element Method
The finite element method was designed to deal with problem with complicated computational regions.
The PDE is first recast into a variational form which essentially forces the mean error to be small everywhere.
The discretization step proceeds by dividing the computational domain into elements of triangular or
rectangular shape. The solution within each element is interpolated with a polynomial of usually low order.
Again, the unknowns are the solution at the collocation points. The CFD community adopted the FEM in the
1980s when reliable methods for dealing with advection dominated problems were devised.
:-
16. Spectral method :-
Both finite element and finite difference methods are low order methods, usually of 2nd − 4th order,
and have local approximation property. By local we mean that a particular collocation point is affected by a
limited number of points around it. In contrast, spectral method have global approximation property. The
interpolation functions, either polynomials or trigonomic functions are global in nature. Their main benefits is
in the rate of convergence which depends on the smoothness of the solution (i.e. how many continuous
derivatives does it admit). For infinitely smooth solution, the error decreases exponentially, i.e. faster than
algebraic. Spectral methods are mostly used in the computations of homogeneous turbulence, and require
relatively simple geometries. Atmospheric model have also adopted spectral methods because of their
convergence properties and the regular spherical shape of their computational domain.
Finite volume method :-
Finite volume methods are primarily used in aerodynamics applications where strong shocks and
discontinuities in the solution occur. Finite volume method solves an integral form of the governing equations
so that local continuity property do not have to hold.
17. Computational cost :-
The CPU time to solve the system of equations differs substantially from method to method. Finite
differences are usually the cheapest on a per grid point basis followed by the finite element method and
spectral method. However, a per grid point basis comparison is a little like comparing apple and oranges.
Spectral methods deliver more accuracy on a per grid point basis than either FEM or FDM. The comparison is
more meaningful if the question is recast as ”what is the computational cost to achieve a given error
tolerance?”. The problem becomes one of defining the error measure which is a complicated task in general
situations.
Forward Euler approximation :-
𝑢𝑛+1 − 𝑢𝑛
∆𝑡
≈ 𝑘𝑢𝑛
Equation is an explicit approximation to the original differential equation since no information about the
unknown function at the future time (n + 1)t has been used on the right hand side of the equation.In order to
derive the error committed in the approximation we rely again on Taylor series.
18. Backward difference :-
This is an example of an implicit method since the unknown u(n + 1) has been used in evaluating the
slope of the solution on the right hand side; this is not a problem to solve for u(n + 1) in this scalar and linear
case. For more complicated situations like a nonlinear right hand side or a system of equations, a nonlinear
system of equations may have to be inverted.
19. Methods of line :-
The method of lines is a technique for solving partial differential equations
(PDEs) in which all but one dimension is discretized. MOL allows
standard, general-purpose methods and software, developed for the
numerical integration of ODEs and DAEs, to be used. A large number of
integration routines have been developed over the years in many different
programming languages, and some have been published as open source
resources. The method of lines most often refers to the construction or
analysis of numerical methods for partial differential equations that
proceeds by first discretizing the spatial derivatives only and leaving the
time variable continuous. This leads to a system of ordinary differential
equations to which a numerical method for initial value ordinary equations
can be applied. The method of lines in this context dates back to at least
the early 1960s. Many papers discussing the accuracy and stability of the
method of lines for various types of partial differential equations have
appeared since.
Method of lines - the example,
which shows the origin of the
name of method.
20. Application to elliptical equations:-
MOL requires that the PDE problem is well-posed as an initial value (Cauchy) problem in at least one
dimension, because ODE and DAE integrators are initial value problem (IVP) solvers. Thus it cannot be used
directly on purely elliptic partial differential equations, such as Laplace's equation. However, MOL has been
used to solve Laplace's equation by using the method of false transients. In this method, a time derivative of
the dependent variable is added to Laplace’s equation. Finite differences are then used to approximate the
spatial derivatives, and the resulting system of equations is solved by MOL. It is also possible to solve elliptical
problems by a semi-analytical method of lines. In this method, the discretization process results in a set of
ODE's that are solved by exploiting properties of the associated exponential matrix. Recently, to overcome the
stability issues associated with the method of false transients, a perturbation approach was proposed which
was found to be more robust than standard method of false transients for a wide range of elliptic PDEs.
21. Multigrid method :-
Multigrid (MG) methods in numerical analysis are algorithms for solving differential equations using a
hierarchy of discretizations. They are an example of a class of techniques called multiresolution methods,
very useful in problems exhibiting multiple scales of behavior. For example, many basic relaxation methods
exhibit different rates of convergence for short- and long-wavelength components, suggesting these different
scales be treated differently, as in a Fourier analysis approach to multigrid. MG methods can be used as
solvers as well as preconditioners. The main idea of multigrid is to accelerate the convergence of a basic
iterative method (known as relaxation, which generally reduces short-wavelength error) by a global correction
of the fine grid solution approximation from time to time, accomplished by solving a coarse problem. The
coarse problem, while cheaper to solve, is similar to the fine grid problem in that it also has short- and long-
wavelength errors. It can also be solved by a combination of relaxation and appeal to still coarser grids. This
recursive process is repeated until a grid is reached where the cost of direct solution there is negligible
compared to the cost of one relaxation sweep on the fine grid. This multigrid cycle typically reduces all error
components by a fixed amount bounded well below one, independent of the fine grid mesh size. The typical
application for multigrid is in the numerical solution of elliptic partial differential equations in two or more
dimensions.
22. Multigrid methods can be applied in combination with any of the common discretization techniques. For
example, the finite element method may be recast as a multigrid method. In these cases, multigrid methods
are among the fastest solution techniques known today. In contrast to other methods, multigrid methods are
general in that they can treat arbitrary regions and boundary conditions. They do not depend on the
separability of the equations or other special properties of the equation. They have also been widely used for
more-complicated non-symmetric and nonlinear systems of equations, like the Lamé equations of elasticity or
the Navier-Stokes equations.
Algorithm :-
There are many variations of multigrid algorithms, but the common features are that a hierarchy of
discretizations (grids) is considered. The important steps are:
1. Smoothing – reducing high frequency errors, for example using a few iterations of the Gauss–Seidel
method.
2. Restriction – down sampling the residual error to a coarser grid.
3. prolongation – interpolating a correction computed on a coarser grid into a finer grid.
24. Computational cost :-
This approach has the advantage over other methods that it often scales linearly with the number of
discrete nodes used. In other words, it can solve these problems to a given accuracy in a number of operations
that is proportional to the number of unknowns. Assume that one has a differential equation which can be
solved approximately (with a given accuracy) on a grid 𝑖with a given grid point density 𝑁𝑖. Assume furthermore
that a solution on any grid 𝑁𝑖may be obtained with a given effort 𝑊𝑖 = 𝜌𝐾𝑁𝑖from a solution on a coarser grid 𝑖 +
1. Here 𝜌 =
𝑁𝑖+1
𝑁𝑖
<1 is the ratio of grid points on "neighboring" grids and is assumed to be constant throughout
the grid hierarchy, and 𝐾is some constant modeling the effort of computing the result for one grid point.
The following recurrence relation is then obtained for the effort of obtaining the solution on grid 𝑘: −
𝑊𝑘 = 𝑊𝑘+1 + 𝜌𝐾𝑁𝑘
And in particular, we find for the finest grid 𝑁1 that
𝑊1 = 𝑊2 + 𝜌𝐾𝑁1
25. Combining these above two expressions (and using 𝑁𝑘 = 𝜌𝑘−1𝑁1 ) gives
𝑊1 = 𝐾𝑁1
𝑝=0
𝑛
𝜌𝑝
Using the geometric series, we then find (for finite 𝑛 )
𝑊1 < 𝐾𝑁1
1
1 − 𝜌
Using the geometric series, we then find (for finite 𝑂 𝑁 time.
26. Multigrid preconditioning :-
A multigrid method with an intentionally reduced tolerance can be used as an efficient preconditioner for
an external iterative solver. The solution may still be obtained in 𝑂 𝑁 time as well as in the case where the
multigrid method is used as a solver. Multigrid preconditioning is used in practice even for linear systems,
typically with one cycle per iteration. Its main advantage versus a purely multigrid solver is particularly clear for
nonlinear problems, e.g., eigenvalue problems.
Generalized multigrid methods :-
Multigrid methods can be generalized in many different ways. They can be applied naturally in a time-
stepping solution of parabolic partial differential equations, or they can be applied directly to time-dependent
partial differential equations. Research on multilevel techniques for hyperbolic partial differential equations is
underway. Multigrid methods can also be applied to integral equations, or for problems in statistical physics.
Other extensions of multigrid methods include techniques where no partial differential equation nor geometrical
problem background is used to construct the multilevel hierarchy. Such algebraic multigrid methods (AMG)
construct their hierarchy of operators directly from the system matrix. In classical AMG, the levels of the
hierarchy are simply subsets of unknowns without any geometric interpretation.
27. (More generally, coarse grid unknowns can be particular linear combinations of fine grid unknowns.) Thus,
AMG methods become black-box solvers for certain classes of sparse matrices. AMG is regarded as
advantageous mainly where geometric multigrid is too difficult to apply, but is often used simply because it
avoids the coding necessary for a true multigrid implementation. While classical AMG was developed first, a
related algebraic method is known as smoothed aggregation (SA). Another set of multiresolution methods is
based upon wavelets. These wavelet methods can be combined with multigrid methods. For example, one use
of wavelets is to reformulate the finite element approach in terms of a multilevel method. Adaptive multigrid
exhibits adaptive mesh refinement, that is, it adjusts the grid as the computation proceeds, in a manner
dependent upon the computation itself. The idea is to increase resolution of the grid only in regions of the
solution where it is needed.
Multigrid in time methods :-
Multigrid methods have also been adopted for the solution of initial value problems. Of particular interest here
are parallel-in-time multigrid methods: in contrast to classical Runge-Kutta or linear multistep methods, they
can offer concurrency in temporal direction. The well known Parareal parallel-in-time integration method can
also be reformulated as a two-level multigrid in time.
28. At last we can say that basically in this topic analyzed the basic techniques for the efficient numerical solution of
problems in science and engineering. Topics spanned root finding, interpolation, approximation of functions,
integration, differential equations and direct and iterative methods in linear algebra.
29. 1. www.wikipedia.org
2. www.khanacademy.com
3. Wiley online library
4. www.byjus.com
5. www.readingfanatic.com
6. Numerical Analysis-by Gupta, Malik and Chauhan.(Krishna prakashan media(p) Ltd)
7. An Introduction to Numerical Analysis-by Kendall E. Atkinson.(Wiley India(p) Ltd)
