Alogrithms Design and Analysis

1. Algorithms
Design and Analysis

2. Course Objectives
• This course introduces students to the analysis and
design of computer algorithms. Upon completion of this
course, students will be able to do the following:
• Analyze the asymptotic performance of algorithms.
• Demonstrate a familiarity with major algorithms and
data structures.
• Apply important algorithmic design paradigms and
methods of analysis.
• Synthesize efficient algorithms in common
engineering design situations.

3. Course Objectives
In this course we will cover the following topics:
 Understand foundations of algorithms &&
design and analysis various variants
algorithms
 Accuracy
 Efficiency
 Comparing efficiencies

4. Algorithms
• The word algorithm comes from the name of a Persian
mathematician Abu Ja’far Mohammed ibn-i Musa al
Khowarizmi.
• In computer science, this word refers to a special
method useable by a computer for solution of a problem.
The statement of the problem specifies in general terms
the desired input/output relationship.
• For example, sorting a given sequence of numbers into
nondecreasing order provides fertile ground for
introducing many standard design techniques and
analysis tools.

5. What is a program?
A program is the expression of an algorithm in a
programming language
a set of instructions which the computer will
follow to solve a problem

6. Problem-Solving (Science and Engineering)
• Analysis
• How does it work?
• Breaking a system down to known components
• How the components relate to each other
• Breaking a process down to known functions
• Synthesis
• Building tools and toys!
• What components are needed
• How the components should be put together
• Composing functions to form a process

7. Problem Solving Using Computers
• Problem:
• Strategy:
• Algorithm:
• Input:
• Output:
• Step:
• Analysis:
• Correctness:
• Time & Space:
• Optimality:
• Implementation:
• Verification:

8. Why need algorithm analysis ?
Writing a working program is not good enough
The program may be inefficient!
If the program is run on a large data set, then
the running time becomes an issue

9. Analyzing Algorithms and Problems
• We analyze algorithms with the intention of
improving them, if possible, and for choosing
among several available for a problem.
• Correctness
• Amount of work done, and space used
• Optimality, Simplicity

10. Correctness can be proved!
• An algorithm consists of sequences of steps
(operations, instructions, statements) for
transforming inputs (preconditions) to outputs
(postconditions)
• Proving
• if the preconditions are satisfied,
• then the postconditions will be true,
• when the algorithm terminates.

11. Amount of work done
• We want a measure of work that tells us
something about the efficiency of the method
used by the algorithm
• independent of computer, programming
language, programmer, and other
implementation details.
• Usually depending on the size of the input

12. Amount of work done
• Counting passes through loops
• Basic Operation
• Identify a particular operation fundamental to the
problem
• the total number of operations performed is roughly
proportional to the number of basic operations
• Identifying the properties of the inputs that affect
the behavior of the algorithm

13. Space Usage
• If memory cells used by the algorithms depends
on the particular input,
• then worst-case and average-case analysis can be
done.
• Time and Space Tradeoff.

14. Optimality “the best possible”
• Each problem has inherent complexity
• There is some minimum amount of work required to
solve it.
• To analyze the complexity of a problem,
• we choose a class of algorithms, based on which
• prove theorems that establish a lower bound on the
number of operations needed to solve the problem.
• Lower bound (for the worst case)

15. Simplicity
• Simplicity in an algorithm is a virtue.

16. Designing Algorithms
• Problem solving using Computer
• Algorithm Design Techniques
• divide-and-conquer
• greedy methods
• depth-first search (for graphs)
• dynamic programming

17. The problem of sorting

18. Analysis of algorithms
The theoretical study of computer-program
performance and resource usage.
What’s more important than performance?
• modularity
• correctness
• maintainability
• functionality
• robustness
• user-friendliness
• programmer time
• simplicity
• extensibility
• reliability

19. Analysis of algorithms
Why study algorithms and performance?
• Algorithms help us to understand scalability.
• Performance often draws the line between what is feasible
and what is impossible.
• Algorithmic mathematics provides a language for talking
about program behavior.
• The lessons of program performance generalize to other
computing resources.
• Speed is fun!

20. Running Time
• The running time depends on the input: an already
sorted sequence is easier to sort.
• Parameterize the running time by the size of the
input, since short sequences are easier to sort than
long ones.
• Generally, we seek upper bounds on the running
time, because everybody likes a guarantee.

21. Kinds of analyses
Worst-case: (usually)
• T(n) = maximum time of algorithm on any input of
size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm over all inputs of
size n.
• Need assumption of statistical distribution of inputs.
Best-case:
• Cheat with a slow algorithm that works fast on some
input.

22. Machine-Independent time
The RAM Model
Machine independent algorithm design depends on a
hypothetical computer called Random Acces Machine (RAM).
Assumptions:
• Each simple operation such as +, -, if ...etc takes exactly
one time step.
• Loops and subroutines are not considered simple
operations.
• Each memory acces takes exactly one time step.

23. Machine-independent time
What is sort’s worst-case time?
• It depends on the speed of our computer,
• relative speed (on the same machine),
• absolute speed (on different machines).
BIG IDEA:
• Ignore machine-dependent constants.
• Look at growth of
“Asymptotic Analysis”


n
n
T as
)
(

24. Asymptotic Analysis
Engineering:
• Drop low-order terms;
• ignore leading constants.
• Example:

25. Asymptotic performance