Time and space complexity


Published on

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Time and space complexity

  1. 1. TIME AND SPACE COMPLEXITYTime ComplexityThe total number of steps involved in a solution to solve a problem is the function of the size of theproblem, which is the measure of that problem’s time complexity.some general order that we can consider (c) < O(log n) < O(n) < O(n log n) < O(nc) < O(cn) < O(n!), where c is some constant.Space ComplexitySpace complexity is measured by using polynomial amounts of memory, with an infinite amount of time.The difference between space complexity and time complexity is that space can be reused.Space complexity is not affected by determinism or nondeterminism. Amount of computer memory required during the program execution, as a function of the input sizeA small amount of space, deterministic machines can simulate nondeterministic machines,where as in time complexity, time increase exponentially in this case. A nondeterministicTM using O(n) space can be changed to a deterministic TM using only O 2(n) space.Complexity: why bother?Estimation/Prediction When you write/run a program, you need to be able to predict its needs, its requirements.Usual requirements - execution time - memory spaceQuantities to estimate execution time  time complexity memory space  space complexityIt is pointless to run a program that requires: - 6TeraB of RAM on a desktop machine; - 10,000 years to run. . .You do not want to wait for an hour: - for the result of your query on Google; - when you are checking your bank account online; - when you are opening a picture file on Photoshop; - etc. It is important to write efficient algorithms
  2. 2. Complexity Classes Deterministic Polynomial TimeP-complete Hardest problems in P solvable on parallel computers Nondeterministic polynomial time and YES answers checkable in polynomial time Nondeterministic polynomial time and NO answers checkable inCo-NP polynomial timeNP- Hardest problems in NPcompleteCo-NP-complete Hardest problems in CO-NPNP-hard At least as hard as NP-complete problemsNC Solvable parallel computation efficiencyPSPACE Polynomial memory with unlimited timePSPACE-complete Hardest problems in PSPACEEXPTIME Exponential timeEXPSPACE Exponential memory with unlimited timeBQP Polynomial time on a quantum computerFind the greatest common divisor (GCD) of two integers, mand n.Program (in C): int gcd(int m, int n) /* precondition: m>0 and n>0.Let g=gcd(m,n). */ { while( m > 0 ) { /* invariant: gcd(m,n)=g */ if( n > m ) { int t = m; m = n; n = t; } /* swap */ /* m >= n > 0 */ m -= n; } return n; }At the start of each iteration of the loop, either n>m or m?n. (i) If m?n, then m is replaced by m-n which is smaller than the previous value of m, and still non-negative. (ii) If n>m, m and n are exchanged, and at the next iteration case (i) will apply.
  3. 3. So at each iteration, max(m,n) either remains unchanged (for just one iteration) or it decreases.This cannot go on for ever because m and n are integers (this fact is important), and eventually alower limit is reached, when m=0 and n=g.So the algorithm does terminate.Good test values would include: • special cases where m or n equals 1, or • m, or n, or both equal small primes 2, 3, 5, …, or • products of two small primes such as p1×p2 and p3×p2, • some larger values, but ones where you know the answers, • swapped values, (x,y) and (y,x), because gcd(m,n)=gcd(n,m).The objective in testing is to "exercise" all paths through the code, in different combinations.We can also consider the best,average and worst cases.Average vs. worst-case complexityDefinition (Worst-case complexity) The worst-case complexity is the complexity of an algorithm when the input is the worst possible with respect to complexity.Definition (Average complexity) The average complexity is the complexity of an algorithm that is averaged over all the possible inputs (assuming a uniform distribution over the inputs). We assume that the complexity of the algorithm is T(i) for an input i. The set of possible inputs of size n is denoted In.Big-O Notation Analysis of AlgorithmsBig Oh Notation- A convenient way of describing the growth rate of a function and hence the time complexity of an algorithm. Let n be the size of the input and f (n), g(n) be positive functions of n. The time efficiency of almost all of the algorithms we have discussed can be characterized by only a few growth rate functions:I. O(l) - constant time This means that the algorithm requires the same fixed number of steps regardless of the size of the task. Examples (assuming a reasonable implementation of the task):
  4. 4. A. Push and Pop operations for a stack (containing n elements); B. Insert and Remove operations for a queue.II. O(n) - linear time This means that the algorithm requires a number of steps proportional to the size of the task. Examples (assuming a reasonable implementation of the task): A. Traversal of a list (a linked list or an array) with n elements; B. Finding the maximum or minimum element in a list, or sequential search in an unsorted list of n elements; C. Traversal of a tree with n nodes; D. Calculating iteratively n-factorial; finding iteratively the nth Fibonacci number.III. O(n2) - quadratic timeThe number of operations is proportional to the size of the task squared.Examples: A. Some more simplistic sorting algorithms, for instance a selection sort of n elements; B. Comparing two two-dimensional arrays of size n by n; C. Finding duplicates in an unsorted list of n elements (implemented with two nested loops).IV. O(log n) - logarithmic timeExamples: A. Binary search in a sorted list of n elements; B. Insert and Find operations for a binary search tree with n nodes; C. Insert and Remove operations for a heap with n nodes.V. O(n log n) - "n log n " time Examples: A. More advanced sorting algorithms - quicksort, mergesortVI. O(an) (a > 1) - exponential timeExamples: A. Recursive Fibonacci implementation B. Towers of Hanoi C. Generating all permutations of n symbolsThe best time in the above list is obviously constant time, and the worst is exponential time which, aswe have seen, quickly overwhelms even the fastest computers even for relatively small n.Polynomial growth (linear, quadratic, cubic, etc.) is considered manageable as compared toexponential growth.Order of asymptotic behavior of the functions from the above list:Using the "<" sign informally, we can say thatO(l) < O(log n) < O(n) < O(n log n) < O(n2) < O(n3) < O(an)
  5. 5. A word about O(log n) growth:As we know from the Change of Base Theorem, for any a, b > 0, and a, b != 1Therefore,loga n = C logb nwhere C is a constant equal to loga b.Since functions that differ only by a constant factor have the same order of growth, O(log2 n) is thesame as O(log n).Therefore, when we talk about logarithmic growth, the base of the logarithm is not important, and wecan say simply O(log n).A word about Big-O when a function is the sum of several terms:If a function (which describes the order of growth of an algorithm) is a sum of several terms, its orderof growth is determined by the fastest growing term. In particular, if we have a polynomialp(n) = aknk + ak-1nk-1 + … + a1n + a0its growth is of the order nk:p(n) = O(nk)Example: