Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Loading in …5
×

# 9 big o-notation

2,236 views

Published on

Published in: Education
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

### 9 big o-notation

1. 1. Big O Notation
2. 2. outline  Performance vs Complexity  Algorithm Efficiency  Big O Notation
3. 3. Performance vs Complexity • Performance:  How much time/memory/disk/… is used by the algorithm  Depends on the machine, compiler, code,… • Complexity:  How do the resource requirements of a program/algorithm scale Complexity affects performance but not the other way around If I have an algorithm with high complexity, it will take longer time to run. However, if I have a very slow program, that does not mean that my algorithm is very complex; that could be all other kind of issues that made the program run slow.
4. 4. Complexity of an Algorithm  We are going to count the number of the steps that are taken during the execution  Assumption (simplification):  Each step costs the same amount  +, -, return a value, access an array element, ….
5. 5. Complexity of an Algorithm (cont.) • Example 1: • private int getLastElement (int [] numbers) • { return numbers[numbers.length -1]; } Number of steps: 4 Constant time function (independent of array size) Notes: Let’s look at each step that need to be executed. It is always 4 steps to access the last element, no matter the number of elements.
6. 6. Complexity of an Algorithm (cont.) • Example 2: • private static int sum(int [] numbers) • { int sum = 0; for (int i = 0; i < number.length; i++){ sum = sum + number[i]; } return sum; } Outside loop: 3 + 2 Number of steps:6n + 5 Linear time functionLoop body: 6 INSTRUCTOR-NOTES:This time we use a loop. We will start by the steps that will be executed only once and the are outside the loop. Then we count the steps that are going to be repeated over and over again.
7. 7. Algorithm Efficiency  Let’s look at the following algorithm for initializing the values in an array: final int N = 500; int [] counts = new int[N]; for (int i=0; i<counts.length; i++) counts[i] = 0;  The length of time the algorithm takes to execute depends on the value of N
8. 8. Algorithm Efficiency (cont.)  In that algorithm, we have one loop that processes all of the elements in the array  Intuitively: – If N was half of its value, we would expect the algorithm to take half the time – If N was twice its value, we would expect the algorithm to take twice the time  That is true and we say that the algorithm efficiency relative to N is linear
9. 9. Algorithm Efficiency (cont.)  Let’s look at another algorithm for initializing the values in a different array: final int N = 500; int [] [] counts = new int[N][N]; for (int i=0; i<counts.length; i++) for (int j=0; j<counts[i].length; j++) counts[i][j] = 0;  The length of time the algorithm takes to execute still depends on the value of N
10. 10. Algorithm Efficiency (cont.)  However, in the second algorithm, we have two nested loops to process the elements in the two dimensional array  Intuitively: – If N is half its value, we would expect the algorithm to take one quarter the time – If N is twice its value, we would expect the algorithm to take quadruple the time  That is true and we say that the algorithm efficiency relative to N is quadratic
11. 11. Big O notation Why do we need Big-O?  In computer science, we use Big-O to classify algorithms based on their scalability.  Basically, it tells you how fast a function grows or declines.  The letter O is used because the rate of growth of a function is also called its order.  It also allows us to estimate the worst case running time of an algorithm as a function of the input size.
12. 12. Big O notation  Examples: – We can say that the first algorithm is O(n) – We can say that the second algorithm is O(n2)  For any algorithm that has a function g(n) of the parameter n that describes its length of time to execute, we can say the algorithm is O(g(n))  We only include the fastest growing term and ignore any multiplying by or adding of constants (which makes sense because those depend on the particular hardware the program is run on)
13. 13. Big O notation (cont.) Growth Functions: Eight functions O(n) that occur frequently in the analysis of algorithms (in order of increasing rate of growth relative to n): – Constant  1 – Logarithmic  log n – Linear  n – Log Linear  n log n – Quadratic  n2 – Cubic  n3 – Exponential  2n – Exhaustive Search  n!
14. 14. Big O notation (cont.) Growth Rates Compared:
15. 15. Big O notation (cont.)
16. 16. Big O notation (cont.)  The growth list is useful because of the following fact:  if a function f(n) is a sum of functions, one of which grows faster than the others, then the faster growing one determines the order of f(n).  Example:  If f(n) = 10 log(n) + 5 (log(n))3 + 7 n + 3 n 2 + 6 n 3,  then f(n) = O(n 3).
17. 17. Big O notation (cont.) Big-O for a Problem:  O(g(n)) for a problem means there is some O(g(n)) algorithm that solves the problem  Don’t assume that the specific algorithm that you are currently using is the best solution for the problem  There may be other correct algorithms that grow at a smaller rate with increasing n  Many times, the goal is to find an algorithm with the smallest possible growth rate
18. 18. Big O notation (cont.) Role of Data Structures:  That brings up the topic of the structure of the data on which the algorithm operates  If we are using an algorithm manually on some amount of data, we intuitively try to organize the data in a way that minimizes the number of steps that we need to take  Publishers offer dictionaries with the words listed in alphabetical order to minimize the length of time it takes us to look up a word
19. 19. Big O notation (cont.) Role of Data Structures:  We can do the same thing for algorithms in our computer programs  Example: Finding a numeric value in a list  If we assume that the list is unordered, we must search from the beginning to the end  On average, we will search half the list  Worst case, we will search the entire list  Algorithm is O(n), where n is size of array
20. 20. Big O notation (cont.) Example: int [] list = {7, 2, 9, 5, 6, 4}; for (int i=0; i<list.length, i++) if (value == list[i]) statement; // found it // didn’t find it
21. 21. Big O notation (cont.)  If we assume that the list is ordered, we can still search the entire list from the beginning to the end to determine if we have a match  But, we do not need to search that way  Because the values are in numerical order, we can use a binary search algorithm  Algorithm is O(logn), where n is size of array
22. 22. Big O notation (cont.) Role of Data Structures:  The difference in the structure of the data between an unordered list and an ordered list can be used to reduce algorithm Big-O  This is the role of data structures and why we study them  We need to be as clever in organizing our data efficiently as we are in figuring out an algorithm for processing it efficiently
23. 23. Big O notation (cont.) Example:  Find an order for  f(x) = 7 x5 + 5 x3 – x + 4  O(x5)
24. 24. Big O notation (cont.) Sorting Algorithms Big-O: http://bigocheatsheet.com/