This course introduces students to analyzing and designing computer algorithms. The course objectives are to analyze asymptotic performance, demonstrate familiarity with major algorithms and data structures, apply algorithm design paradigms and analysis methods, and synthesize efficient algorithms for engineering problems. Topics covered include foundations of algorithms, accuracy, efficiency, comparing efficiencies, and understanding various algorithm variants and their design and analysis.

1. Algorithms
Design and Analysis

2. Course Objectives
• This course introduces students to the analysis and
design of computer algorithms. Upon completion of this
course, students will be able to do the following:
• Analyze the asymptotic performance of algorithms.
• Demonstrate a familiarity with major algorithms and
data structures.
• Apply important algorithmic design paradigms and
methods of analysis.
• Synthesize efficient algorithms in common
engineering design situations.

3. Course Objectives
In this course we will cover the following topics:
 Understand foundations of algorithms &&
design and analysis various variants
algorithms
 Accuracy
 Efficiency
 Comparing efficiencies

4. Algorithms
• The word algorithm comes from the name of a Persian
mathematician Abu Ja’far Mohammed ibn-i Musa al
Khowarizmi.
• In computer science, this word refers to a special
method useable by a computer for solution of a problem.
The statement of the problem specifies in general terms
the desired input/output relationship.
• For example, sorting a given sequence of numbers into
nondecreasing order provides fertile ground for
introducing many standard design techniques and
analysis tools.

5. What is a program?
A program is the expression of an algorithm in a
programming language
a set of instructions which the computer will
follow to solve a problem

6. Problem-Solving (Science and Engineering)
• Analysis
• How does it work?
• Breaking a system down to known components
• How the components relate to each other
• Breaking a process down to known functions
• Synthesis
• Building tools and toys!
• What components are needed
• How the components should be put together
• Composing functions to form a process

7. Problem Solving Using Computers
• Problem:
• Strategy:
• Algorithm:
• Input:
• Output:
• Step:
• Analysis:
• Correctness:
• Time & Space:
• Optimality:
• Implementation:
• Verification:

8. Why need algorithm analysis ?
Writing a working program is not good enough
The program may be inefficient!
If the program is run on a large data set, then
the running time becomes an issue

9. Analyzing Algorithms and Problems
• We analyze algorithms with the intention of
improving them, if possible, and for choosing
among several available for a problem.
• Correctness
• Amount of work done, and space used
• Optimality, Simplicity

10. Correctness can be proved!
• An algorithm consists of sequences of steps
(operations, instructions, statements) for
transforming inputs (preconditions) to outputs
(postconditions)
• Proving
• if the preconditions are satisfied,
• then the postconditions will be true,
• when the algorithm terminates.

11. Amount of work done
• We want a measure of work that tells us
something about the efficiency of the method
used by the algorithm
• independent of computer, programming
language, programmer, and other
implementation details.
• Usually depending on the size of the input

12. Amount of work done
• Counting passes through loops
• Basic Operation
• Identify a particular operation fundamental to the
problem
• the total number of operations performed is roughly
proportional to the number of basic operations
• Identifying the properties of the inputs that affect
the behavior of the algorithm

13. Space Usage
• If memory cells used by the algorithms depends
on the particular input,
• then worst-case and average-case analysis can be
done.
• Time and Space Tradeoff.

14. Optimality “the best possible”
• Each problem has inherent complexity
• There is some minimum amount of work required to
solve it.
• To analyze the complexity of a problem,
• we choose a class of algorithms, based on which
• prove theorems that establish a lower bound on the
number of operations needed to solve the problem.
• Lower bound (for the worst case)

15. Simplicity
• Simplicity in an algorithm is a virtue.

16. Designing Algorithms
• Problem solving using Computer
• Algorithm Design Techniques
• divide-and-conquer
• greedy methods
• depth-first search (for graphs)
• dynamic programming

17. The problem of sorting

18. Analysis of algorithms
The theoretical study of computer-program
performance and resource usage.
What’s more important than performance?
• modularity
• correctness
• maintainability
• functionality
• robustness
• user-friendliness
• programmer time
• simplicity
• extensibility
• reliability

19. Analysis of algorithms
Why study algorithms and performance?
• Algorithms help us to understand scalability.
• Performance often draws the line between what is feasible
and what is impossible.
• Algorithmic mathematics provides a language for talking
about program behavior.
• The lessons of program performance generalize to other
computing resources.
• Speed is fun!

20. Running Time
• The running time depends on the input: an already
sorted sequence is easier to sort.
• Parameterize the running time by the size of the
input, since short sequences are easier to sort than
long ones.
• Generally, we seek upper bounds on the running
time, because everybody likes a guarantee.

21. Kinds of analyses
Worst-case: (usually)
• T(n) = maximum time of algorithm on any input of
size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm over all inputs of
size n.
• Need assumption of statistical distribution of inputs.
Best-case:
• Cheat with a slow algorithm that works fast on some
input.

22. Machine-Independent time
The RAM Model
Machine independent algorithm design depends on a
hypothetical computer called Random Acces Machine (RAM).
Assumptions:
• Each simple operation such as +, -, if ...etc takes exactly
one time step.
• Loops and subroutines are not considered simple
operations.
• Each memory acces takes exactly one time step.

23. Machine-independent time
What is sort’s worst-case time?
• It depends on the speed of our computer,
• relative speed (on the same machine),
• absolute speed (on different machines).
BIG IDEA:
• Ignore machine-dependent constants.
• Look at growth of
“Asymptotic Analysis”


n
n
T as
)
(

24. Asymptotic Analysis
Engineering:
• Drop low-order terms;
• ignore leading constants.
• Example:

25. Asymptotic performance