Your SlideShare is downloading. ×
0
Asymptotes and Algorithms        By Gary Short      Gibraltar Software                            1
Agenda•   Introduction•   Performance, does it matter?•   How do we measure performance?•   Analysis of Insertion Sort•   ...
Introduction•   Gary Short•   Head of Gibraltar Labs•   C# MVP•   gary.short@gibraltarsoftware.com•   @garyshort•   http:/...
Performance – Does it Matter?Performance is the most important thing in  software engineering today...                    ...
... Apart from everything else!                                  5
So Why Bother About Performance?                                   6
How do we Measure Performance?• What do we care about?  – Memory?  – Bandwidth?  – Computational time?                    ...
We Need a Model to Work With• RAM Model  – Arithmetic – add, subtract, etc  – Data movement – load, copy, store  – Control...
Analysis of Insertion SortInsertionSort(A)  for j = 2 to A.length      key=[Aj]      i=j-1      while i > 0 and A[i] > key...
That Makes no Sense, Show me!                                10
So What’s The Running Time?                              11
Sum Running Time for Each Statement...T(n) = c1n+c2(n-1)+c3(n-1)+c4 sum(tj) j=2..n+c5  sum(tj-1) j=2..n+c6sum(tj-1) j=2..n...
Best Case Running TimeIf the input (A) is already sorted then...A[i] <= key when has initial value of j-1 thus tj=1.And so...
14
Side Note: No One Cares About Best Case                                          15
Worst Case ScenarioIf the input (n) is in reverse sort order then...We have to compare each A[j] with each   element in th...
17
In Short...In worst case insertion sort sucks!                                         18
Man That Was a Lot of Maths!                               19
Simplifying Things With Asymptotic Notation• Asymptotic notation characterises functions  by their growth rates• Functions...
How Does That Help Us?Let’s say we have a function with running timeT(n) = 4n^2 - 2n + 2If n = 500 then4n^2 is 1000 times ...
A Short Note on The Abuse of “=“If T(n) = 4n^2 -2n +2Then saying T(n) = O(n^2) is not strictly correctRather T(n) is in th...
So Back to Insertion SortSo now we can say of Insertion Sort that...Best case it’s O(n)And worst case it’s O(n^2)And since...
Designing Algorithms  So can we do better?                         24
Optimizing Algorithms is Child’s Play• Sit at table• Foreach item in itemsOnPlate  – Eat item• Wait(MealComplete)• Foreach...
Child Will Optimize To…•   Pause Game•   Set Speed = MaxInt•   Run to table•   Take sliceBread(1)•   Foreach item on Plate...
Divide And Conquer• Divide  – Divide the problem into sub problems• Conquer  – Solve the sub problems recursively• Combine...
Merge Sort• Divide  – Divide the n elements into two n/2 element arrays• Conquer  – Sort the two arrays recursively• Combi...
Analysis of Merge SortMergeSort(A,p,r)  if(p<r)       q = [(p+r)/2]       MergeSort(A,p,q)       MergeSort(A,q+1,r)       ...
Dancers, or it Never Happened!!                                  30
So What’s The Running Time?In the general case...If the divide step yields ‘a’ sub problemsEach 1/b the size of the origin...
Apply That to Merge Sort...• Divide  – Computes the middle of the subarray, taking    constant time so, D(n) = O(1)• Conqu...
Solve The Recurrence Using The Master MethodFor a Recurrence in the formT(n) = aT(n/b) + f(n)ThenIf f(n) = O(nlogba-k) the...
What?!• More simply we are comparing f(n) with the  function n log ba and intuitively  understanding that the bigger of th...
And So...• With Merge Sort we are in the third case of  the Master Method thus...• T(n) = O(n log n)• Which is much better...
36
What We Learned•   Performance is important•   Therefore algorithmic optimization is too•   We have a model to benchmark• ...
38
Questions?             39
Upcoming SlideShare
Loading in...5
×

Algorithms - Rocksolid Tour 2013

484

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
484
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Algorithms - Rocksolid Tour 2013"

  1. 1. Asymptotes and Algorithms By Gary Short Gibraltar Software 1
  2. 2. Agenda• Introduction• Performance, does it matter?• How do we measure performance?• Analysis of Insertion Sort• Simplifying things with asymptotic notation• Designing algorithms• Solving recurrences• Questions. 2
  3. 3. Introduction• Gary Short• Head of Gibraltar Labs• C# MVP• gary.short@gibraltarsoftware.com• @garyshort• http://www.facebook.com/theothergaryshort 3
  4. 4. Performance – Does it Matter?Performance is the most important thing in software engineering today... 4
  5. 5. ... Apart from everything else! 5
  6. 6. So Why Bother About Performance? 6
  7. 7. How do we Measure Performance?• What do we care about? – Memory? – Bandwidth? – Computational time? 7
  8. 8. We Need a Model to Work With• RAM Model – Arithmetic – add, subtract, etc – Data movement – load, copy, store – Control – branching, subroutine call, return – Data – Integers, floats• Instruction are run in series – And take constant time • Not really, but shhh! –Ed.  8
  9. 9. Analysis of Insertion SortInsertionSort(A) for j = 2 to A.length key=[Aj] i=j-1 while i > 0 and A[i] > key A[i+1] = A[i] i=i-1 A[i+1] = key 9
  10. 10. That Makes no Sense, Show me! 10
  11. 11. So What’s The Running Time? 11
  12. 12. Sum Running Time for Each Statement...T(n) = c1n+c2(n-1)+c3(n-1)+c4 sum(tj) j=2..n+c5 sum(tj-1) j=2..n+c6sum(tj-1) j=2..n+c7(n-1) 12
  13. 13. Best Case Running TimeIf the input (A) is already sorted then...A[i] <= key when has initial value of j-1 thus tj=1.And so...T(n) = c1n+c2(n-1)+c3(n-1)+c4(n-1)+c7(n-1)= (c1+c2+c3+c4+c7)n-(c2+c3+c4+c7)Which can be expressed as an+b for constants a and b that depend on ciSo T(n) is a linear function of n 13
  14. 14. 14
  15. 15. Side Note: No One Cares About Best Case 15
  16. 16. Worst Case ScenarioIf the input (n) is in reverse sort order then...We have to compare each A[j] with each element in the sub array A[1..j-1].And so...T(n) = (c4/2+c5/2+c6/2)n^2 +(c1 +c2+c3+c4/2- c5/2-c6/2+c7)n-(c2+c3+c4+c7)Which can be expressed as an^2 + bn + cSo T(n) is a quadratic function of n 16
  17. 17. 17
  18. 18. In Short...In worst case insertion sort sucks!  18
  19. 19. Man That Was a Lot of Maths! 19
  20. 20. Simplifying Things With Asymptotic Notation• Asymptotic notation characterises functions by their growth rates• Functions with the same growth rates have the same Asymptotic notation 20
  21. 21. How Does That Help Us?Let’s say we have a function with running timeT(n) = 4n^2 - 2n + 2If n = 500 then4n^2 is 1000 times bigger than 2nSo...We can ignore smaller order terms and coefficientsT(n) = 4n^2 -2n +2 can be written O(n) = n^2 21
  22. 22. A Short Note on The Abuse of “=“If T(n) = 4n^2 -2n +2Then saying T(n) = O(n^2) is not strictly correctRather T(n) is in the set O(n^2) and the above should be read as T(n) is O(n^2) and not T(n) equals O(n^2)But really on Maths geeks care – Ed.  22
  23. 23. So Back to Insertion SortSo now we can say of Insertion Sort that...Best case it’s O(n)And worst case it’s O(n^2)And since we only care about worst case...We say that Insertion Sort has O(n^2)Which sucks! – Ed.  23
  24. 24. Designing Algorithms So can we do better? 24
  25. 25. Optimizing Algorithms is Child’s Play• Sit at table• Foreach item in itemsOnPlate – Eat item• Wait(MealComplete)• Foreach dish in dishesUsed – WashDish – DryDish• Resume Play 25
  26. 26. Child Will Optimize To…• Pause Game• Set Speed = MaxInt• Run to table• Take sliceBread(1)• Foreach item on Plate – Place item on bread• Take sliceBread(2)• Run Outside• Resume Game 26
  27. 27. Divide And Conquer• Divide – Divide the problem into sub problems• Conquer – Solve the sub problems recursively• Combine – Add the solutions to the sub problems into the solution for the original problem. 27
  28. 28. Merge Sort• Divide – Divide the n elements into two n/2 element arrays• Conquer – Sort the two arrays recursively• Combine – Merge the two sorted arrays to produce the answer. 28
  29. 29. Analysis of Merge SortMergeSort(A,p,r) if(p<r) q = [(p+r)/2] MergeSort(A,p,q) MergeSort(A,q+1,r) Merge(A,p,q,r)Initial call MergeSort(A,1,A.length) 29
  30. 30. Dancers, or it Never Happened!! 30
  31. 31. So What’s The Running Time?In the general case...If the divide step yields ‘a’ sub problemsEach 1/b the size of the originalIt takes T(n/b) time to solve one problem of n/b sizeSo it takes aT(n/b) to solve ‘a’ of themThen, if it takes D(n) time to divide the problemAnd C(n) time to combine the resultsThen we get the recurrence...T(n) = aT(n/b) + D(n) + C(n). 31
  32. 32. Apply That to Merge Sort...• Divide – Computes the middle of the subarray, taking constant time so, D(n) = O(1)• Conquer – Recursively solve two sub problems each of size n/2 contributing 2T(n/2) to the running time• Combine – Merge procedure O(n)• Giving us a recurrence of 2T(n/2)+O(n) 32
  33. 33. Solve The Recurrence Using The Master MethodFor a Recurrence in the formT(n) = aT(n/b) + f(n)ThenIf f(n) = O(nlogba-k) then T(n) = O(nlogba)If f(n) = O(nlogba) then T(n) = O(nlogba log n)if f(n) = Omega(n log b a+k) and if af(n/b) <= cf(n) then T(n) = O(f(n)) 33
  34. 34. What?!• More simply we are comparing f(n) with the function n log ba and intuitively understanding that the bigger of the two determines the solution to the recurrence. 34
  35. 35. And So...• With Merge Sort we are in the third case of the Master Method thus...• T(n) = O(n log n)• Which is much better than the O(n^2) of Insertion Sort 35
  36. 36. 36
  37. 37. What We Learned• Performance is important• Therefore algorithmic optimization is too• We have a model to benchmark• And a syntax• Divide and conquer• Master Method• Other resources. 37
  38. 38. 38
  39. 39. Questions? 39
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×