Algorithem complexity in data sructure

501 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
501
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Algorithem complexity in data sructure

  1. 1. Algorithmsand DataStructuresMarcinSydowAlgorithms and Data StructuresComplexity of AlgorithmsMarcin Sydow
  2. 2. Algorithmsand DataStructuresMarcinSydowDesired Properties of a Good AlgorithmAny good algorithm should satisfy 2 obvious conditions:1 compute correct (desired) output (for the given problem)2 be eective (fast)ad. 1) correctness of algorithmad. 2) complexity of algorithmComplexity of algorithm measures how fast is the algorithm(time complexity) and what amount of memory it uses(space complexity) - time and memory - 2 basic resources incomputations
  3. 3. Algorithmsand DataStructuresMarcinSydowExample - the Search ProblemProblem of searching a key in an arrayWhat does the amount of work of this algorithm depend on?find(arr, len, key)Specication:input: arr - array of integers, len - its length, key - integer to be foundoutput: integer 0 ≤ i len being the index in arr, under which the key isstored
  4. 4. Algorithmsand DataStructuresMarcinSydowExample - the Search ProblemProblem of searching a key in an arrayWhat does the amount of work of this algorithm depend on?find(arr, len, key)Specication:input: arr - array of integers, len - its length, key - integer to be foundoutput: integer 0 ≤ i len being the index in arr, under which the key isstored (is it a complete/clear specication?)
  5. 5. Algorithmsand DataStructuresMarcinSydowExample - the Search ProblemProblem of searching a key in an arrayWhat does the amount of work of this algorithm depend on?find(arr, len, key)Specication:input: arr - array of integers, len - its length, key - integer to be foundoutput: integer 0 ≤ i len being the index in arr, under which the key isstored (is it a complete/clear specication?) or the value of -1 when thereis no specied key in (rst len positions of) the array
  6. 6. Algorithmsand DataStructuresMarcinSydowExample - the Search ProblemProblem of searching a key in an arrayWhat does the amount of work of this algorithm depend on?find(arr, len, key)Specication:input: arr - array of integers, len - its length, key - integer to be foundoutput: integer 0 ≤ i len being the index in arr, under which the key isstored (is it a complete/clear specication?) or the value of -1 when thereis no specied key in (rst len positions of) the arraycode:find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}What does the amount of work of this algorithm depend on?
  7. 7. Algorithmsand DataStructuresMarcinSydowThe speed of algorithmHow to measure how fast (or slow) an algorithm is?There are 2 issues to be considered when designing such ameasure:1 independence on any programming language (andhardware/software platform)2 maximum independence on particular input dataIt should be an internal property of the algorithm itselfAny idea?
  8. 8. Algorithmsand DataStructuresMarcinSydowThe speed of algorithmHow to measure how fast (or slow) an algorithm is?There are 2 issues to be considered when designing such ameasure:1 independence on any programming language (andhardware/software platform)2 maximum independence on particular input dataIt should be an internal property of the algorithm itselfAny idea? Count basic operations of the algorithm
  9. 9. Algorithmsand DataStructuresMarcinSydowDominating OperationsSimplication: it is not necessary to count all the operations - itis enough to count the representative onesBefore doing a complexity analysis 2 steps must be done:1 determine the dominating operation set2 observe what (in input) inuences the number ofdominating operations (data size)Dominating operations are those which cover the amount ofwork which is proportional to the whole amount of work of thealgorithm (they well represent the whole)
  10. 10. Algorithmsand DataStructuresMarcinSydowExample - determining operating operationsWhat can be the dominating operation set in the followingalgorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}assignment i = 0 ?
  11. 11. Algorithmsand DataStructuresMarcinSydowExample - determining operating operationsWhat can be the dominating operation set in the followingalgorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}assignment i = 0 ? nocomparison i len ?
  12. 12. Algorithmsand DataStructuresMarcinSydowExample - determining operating operationsWhat can be the dominating operation set in the followingalgorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}assignment i = 0 ? nocomparison i len ? yescomparison arr[i] == key ?
  13. 13. Algorithmsand DataStructuresMarcinSydowExample - determining operating operationsWhat can be the dominating operation set in the followingalgorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}assignment i = 0 ? nocomparison i len ? yescomparison arr[i] == key ? yesboth the above?
  14. 14. Algorithmsand DataStructuresMarcinSydowExample - determining operating operationsWhat can be the dominating operation set in the followingalgorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}assignment i = 0 ? nocomparison i len ? yescomparison arr[i] == key ? yesboth the above? yesreturn statement return i ?
  15. 15. Algorithmsand DataStructuresMarcinSydowExample - determining operating operationsWhat can be the dominating operation set in the followingalgorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}assignment i = 0 ? nocomparison i len ? yescomparison arr[i] == key ? yesboth the above? yesreturn statement return i ? noindex increment i++ ?
  16. 16. Algorithmsand DataStructuresMarcinSydowExample - determining operating operationsWhat can be the dominating operation set in the followingalgorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}assignment i = 0 ? nocomparison i len ? yescomparison arr[i] == key ? yesboth the above? yesreturn statement return i ? noindex increment i++ ? yes
  17. 17. Algorithmsand DataStructuresMarcinSydowExample, cont. - determining the data sizeWhat is the data size in the following algorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}
  18. 18. Algorithmsand DataStructuresMarcinSydowExample, cont. - determining the data sizeWhat is the data size in the following algorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}Data size: length of array arr
  19. 19. Algorithmsand DataStructuresMarcinSydowExample, cont. - determining the data sizeWhat is the data size in the following algorithm?find(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}Data size: length of array arrHaving determined the dominating operation and data sizewe can determine time complexity of the algorithm
  20. 20. Algorithmsand DataStructuresMarcinSydowTime Complexity of AlgorithmDenitionTime Complexity of Algorithm is the number of dominatingoperations executed by the algorithm as the function of datasize.Time complexity measures the amount of work done by thealgorithm during solving the problem in the way which isindependent on the implementation and particular input data.The lower time complexity the faster algorithm
  21. 21. Algorithmsand DataStructuresMarcinSydowExample - time complexity of algorithmfind(arr, len, key){i = 0while(i len){if(arr[i] == key)return ii++}return -1}Assume:dominating operation: comparison arr[i] == keydata size: the length of array (denote by n)Thus, the number of dominating operations executed by thisalgorithm ranges:from 1 (the key was found under the rst index)to n (the key is absent or under the last index)There is no single function which could express the dependence of the number ofexecuted dominating operations on the data size for this algorithm.
  22. 22. Algorithmsand DataStructuresMarcinSydowPessimistic Time Complexitylets assume the following denotations:n - data sizeDn - the set of all possible input datasets of size nt(d) - the number of dominating operations for dataset d (of size n)(d ∈ Dn)DenitionPessimistic Time Complexity of algorithm:W(n) = sup{t(d) : d ∈ Dn}(W(n) - Worst)Pessimistic Time Complexity expresses the number ofdominating operations in the worst case of input data of size nE.g. for our example the pessimistic time complexity is given bythe formula:
  23. 23. Algorithmsand DataStructuresMarcinSydowPessimistic Time Complexitylets assume the following denotations:n - data sizeDn - the set of all possible input datasets of size nt(d) - the number of dominating operations for dataset d (of size n)(d ∈ Dn)DenitionPessimistic Time Complexity of algorithm:W(n) = sup{t(d) : d ∈ Dn}(W(n) - Worst)Pessimistic Time Complexity expresses the number ofdominating operations in the worst case of input data of size nE.g. for our example the pessimistic time complexity is given bythe formula:W(n) = n
  24. 24. Algorithmsand DataStructuresMarcinSydowAverage Time Complexity of Algorithmlets assume the following denotations:n - data sizeDn - the set of all possible input datasets of size nt(d) - the number of dominating operations for dataset d (of size n) (d ∈ Dn)Xn - random variable, its value is t(d) for d ∈ Dnpnk - probability distribution of the random variable Xn (i.e. the probability thatfor input data of size n the algorithm will execute k dominating operations(k ≥ 0))DenitionAverage Time Complexity of Algorithm:A(n) = k≥0pnk · k = P(Xn = k) · k(expected value of the random variable representing the numberof dominating operations)(A(n) Average)
  25. 25. Algorithmsand DataStructuresMarcinSydowExample - Determining the Average TimeComplexityLets determine the average time complexity for our exemplaryalgorithm (nd)First, we have to assume some probabilistic model of inputdata (i.e. the probabilistic distribution of possible inputdatasets)Lets make a simplistic assumption: the key to be found occursexactly once in array and with the same probability on eachindex (uniform distribution) (∀0≤knP(Xn = k) = 1/n)Thus:A(n) = k≥0P(Xn = k) · k = 0≤kn 1/n · k = n+12
  26. 26. Algorithmsand DataStructuresMarcinSydowSpace Complexity of AlgorithmDenitionSpace Complexity of Algorithm: S(n) is the number of unitsof memory used by algorithm as a function of data sizeThis characteristic is more dependent on particular platform than time complexity.As a memory unit one can consider the machine word.Note:We will assume, that the memory used for keeping the inputdata is not considered because usually arrays (and othercompound types) are passed as arguments to functions byreference, which does not involve much memoryIn our example space complexity is constant - because it consumes memory onlyfor a single variable (plus some xed number of additional temporal variables),independently on the input data size: S(n) = const
  27. 27. Algorithmsand DataStructuresMarcinSydowOmitting Unimportant DetailsThe real time spent by an implementation of the algorithmmay dier between particular platforms by a constantmultiplicative factor. (e.g. CPU speed)Thus, it would be very useful to have a notation allowing forexpressing the complexity functions with neglectingunimportant details (as multiplicative or additive constant, forexample)E.g. for the following function:A(n) = 2.1 · n − 1The most important information is that it is a linear function -its rank of complexity is linearDoes such a notation exist?
  28. 28. Algorithmsand DataStructuresMarcinSydowAsymptotic Notation - Big OThe notation is called asymptotic notation.There are a couple of avours. The most common is big O:DenitionThe function g(n) is the upper bound of rank of order of thefunction f(n):f (n) = O(g(n)) ⇔ ∃c0∃n0∀n≥n0f (n) ≤ c · g(n)The O() notation intuitively corresponds to the ≤ symbol (interms of ranks of orders of functions).E.g. the fact that W(n) of our exemplary algorithm has anupper bound of the linear rank can be noted as:W(n) = n+12= O(n)The constant space complexity S(n) of that algorithm can beexpressed with the following special notation:S(n) = O(1)
  29. 29. Algorithmsand DataStructuresMarcinSydowAsymptotic Notation - Big ThetaAnother important avour of asymptotic notation is big Theta:DenitionThe function f(n) has the same rank of order as the functiong(n): f (n) = Θ(g(n)) ⇔ f (n) = O(g(n)) ∧ g(n) = O(f (n))The Θ() notation intuitively corresponds to the = symbol (interms of ranks of orders of functions).Notice, that Θ() is dened with the use of O(), similarly as = symbol can bedened with the use of ≤ symbol.E.g. the expression: f (n) = n2+ n − 3 = Θ(n2)reads as the n2+ n − 3 function is of square rank of order.
  30. 30. Algorithmsand DataStructuresMarcinSydowOther Flavours of Asymptotic NotationWe have 5 relation symbols for comparing numbers: = ≤ ≥ In total, we also have 5 analogous symbols for comparing ranksof functions:1 Θ - =2 O - ≤3 Ω - ≥4 o - 5 ω - (in general, a capital letter denotes non-sharp inequality and lowercase denotesa sharp one)E.g.:W(n)=o(n) (lowercase o)means: the rank of function W(n) is lower than linear
  31. 31. Algorithmsand DataStructuresMarcinSydowSome Remarks on Using the Asymptotic NotationNotice: in expressions like f(n)=O(g(n)) the = has a specialmeaning - it does not represent the normal equality. Theexpression has its meaning only as a whole.E.g. it does not make sense to use asymptotic notation as therst expression on the left-hand side of the = symbol.E.g. expressions like O(f(n)) = n or O(f(n)) = O(g(n)) donot make any senseBesides the standard usage of the asymptotic notation on theright-hand side of the = symbol, it can be also used in thefollowing way:f(n) = g(n) + O(h(n))Which means: f(n) - g(n) = O(h(n))(the ranks of functions f and g dier at most by a rank offunction h)
  32. 32. Algorithmsand DataStructuresMarcinSydowRemarks: Comparing Ranks of FunctionsSometimes the following technique is useful.Ranks of some 2 functions f(n) and g(n) can be compared bycomputing the following limit:limn→∞f(n)g(n)there are 3 possible cases for the limit:1 ∞ - in that case f(n)=ω(g(n)) (f has higher rank)2 a positive constant - in that case f(n)=Θ(g(n)) (the sameranks)3 zero - in that case f(n)=o(g(n)) (lowercase o) (g hashigher rank)
  33. 33. Algorithmsand DataStructuresMarcinSydowThe Most Common Ranks of Functionsconstant (e.g. S(n) = 3 = Θ(1))logarithmic (e.g. W(n) = 2 + lg2n = Θ(log(n)))linear (e.g. A(n) = 2n + 1 = Θ(n))linear-logarithmic (e.g.A(n) = 1.44 · nlog(n) = Θ(nlog(n)))square (e.g. W(n) = n2+ 4 = Θ(n2))cubic (e.g. A(n) = Θ(n3))sub-exponential (e.g. A(n) = Θ(nlog(n)))exponential (e.g. A(n) = Θ(2n))factorial (e.g. W(n) = Θ(n!))In simplication: in practise, an over-polynomial rank of time complexity isconsidered as unacceptably highIn case of space complexity, even linear rank is considered as very high
  34. 34. Algorithmsand DataStructuresMarcinSydowQuestions/Problems:How to measure the speed of algorithmWhat 2 things should be determined before startingassessing the time complexity of an algorithmWhat is a dominating operationDenition of Time Complexity of AlgorithmDenition of Space Complexity of AlgorithmDenition of Pessimistic Time ComplexityDenition of Average Time ComplexityBe able to determine time complexity for simple algorithmsWhat is the purpose of the asymptotic notationDenition and interpretation of the O() notationDenitions (and interpretations) of the other types ofasymptotic notationsAbility to express rank of a given function with theasymptotic notation
  35. 35. Algorithmsand DataStructuresMarcinSydowThank you for your attention

×