2. What is Numerical Analysis
NA is the study of algorithms that use numerical
approximation (as opposed to general symbolic
representation) for the problems of mathematical analysis
(Wikipedia)
• Before the advent of modern
computers numerical methods
often depended on hand
interpolation on printed
tables.
Babylonian clay tablet to
calculate 2 (1600 BC)
3. History
• As old as human civilization
– A root-finding method for solving a simple equation in The
Rhind Mathematical Papyrus (Egypt, ~1650 BC).
– Archimedes (212 BC)’s “method of exhaustion” for calculating
lengths, areas, and volumes.
• Modern Applications
– Logarithms (Napier in 1614)
a simple way of doing
arithmetic operations (addition, mult., div., exponents.)
• Numerical alternatives to Calculus problems (Newton,
Leibnitz)
– Many other scientist contributed later on (Euler, Lagrange)
– Approximation techniques, numerical integration, solution of
differential equations, etc.
Applications in many different fields
4. • We use numerical methods when there is no explicit
solution. We develop approximate solutions.
• More computer power better numerical models
• Growth in power and availability of digital computers in
the last 50 years led to an increasing use of sophisticated
mathematical models in sciences and engineering.
Scientific Computing; Computational Sciences.
• Computer arithmetic
– Historically varied among different computer manufacturers
problems problems of porting a software between different
computers
– Standardized by IEEE around 90’s standards for computer
floating-point arithmetic.
5. Some Examples
• Natural Sciences
– Predicting climate change
– Simulating a carbon nanotube
• Social Sciences
– Statistical modeling
• Engineering
– Designing an airplane
– Building modern structures
– Computer Aided Design (CAD)
• Medicine
– Simulating a protein
– Simulating reaction of chemicals
• Business
– Optimization (inventory control, scheduling, storage, etc...)
• Many thousands of other applications...
6. Major concerns
• Replacement of the problem that cannot be solved
directly with a “nearby problem” which can be solved
more easily.
• Widespread use of the language of linear algebra.
• Error, its size and its analytical form; improve the
convergence behavior of the numerical solution.
• Stability (sensitivity of the solution to small changes in
the data or the parameters). Stability must be
established for most real life problems (unless it is a
chaotic system).
• Precision of the computer arithmetic, especially a
concern for large size numerical algebra solutions.
• Efficiency of the algorithms. What is the time (cost) of
solution of n equations in Ax=b?
7. Numerical Linear Algebra
• Solution of System of Equations (Ax=b)
– Direct methods (e.g. Gaussian elimination)
– Indirect (iterative) methods
• Errors in calculation of x
– Rounding errors
arithmetic.
finite length of numbers in computer
• Non-linear problems
– Reducing it to a sequence of a linear problem
– Apply linear solution techniques
• Some examples
– Optimization (Optimum x values for min(f(x))
– Rootfinding (Approximate the function near the root)
8. Approximation Theory
• Approximation for evaluating f(x) using basic
arithmetic operations (addition, subtraction,
multiplication, division) and comparison operations
(true/false)
– Using four basic operations, find polynomial p(x)
p(x)=a0 + a1x + .. + anxn
– Rational functions (polyn/polyn)
– All other functions ( x, 2x) must be reduced to polynomials
or rational functions
• Only finite number of operations are possible.
• All function evaluations on calculators and
computers are performed in this manner.
9. Interpolation
• Interpolation is also a method of approximation
– Finding a polynomial that satisfies a set of points
– One can also use rational functions, trigonometric
polynomials, and spline functions.
• Numerical Integration and Differentiation
– Approximation of integrals and derivatives are based on
interpolation.
– Replace the function f(x) by an interpolating (a simpler)
function p(x)
– Integrate or differentiate p(x)
10. Solving Differential and Integral Equations
• Approximation theory + Solution of linear/non-linear
systems (often very large)
• Most mathematical models in natural sciences and
engineering are based on ODE and PDE.
– Finite element method (Replace the unknown function by a
simpler function, e.g. a polynomial)
– Finite difference methods (Solving the problem in discrete
points, and approximating derivatives and integrals in
discrete points)
