Upcoming SlideShare
×

# Algorithms - Rocksolid Tour 2013

662 views
593 views

Published on

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
662
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
5
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Algorithms - Rocksolid Tour 2013

1. 1. Asymptotes and Algorithms By Gary Short Gibraltar Software 1
2. 2. Agenda• Introduction• Performance, does it matter?• How do we measure performance?• Analysis of Insertion Sort• Simplifying things with asymptotic notation• Designing algorithms• Solving recurrences• Questions. 2
3. 3. Introduction• Gary Short• Head of Gibraltar Labs• C# MVP• gary.short@gibraltarsoftware.com• @garyshort• http://www.facebook.com/theothergaryshort 3
4. 4. Performance – Does it Matter?Performance is the most important thing in software engineering today... 4
5. 5. ... Apart from everything else! 5
6. 6. So Why Bother About Performance? 6
7. 7. How do we Measure Performance?• What do we care about? – Memory? – Bandwidth? – Computational time? 7
8. 8. We Need a Model to Work With• RAM Model – Arithmetic – add, subtract, etc – Data movement – load, copy, store – Control – branching, subroutine call, return – Data – Integers, floats• Instruction are run in series – And take constant time • Not really, but shhh! –Ed.  8
9. 9. Analysis of Insertion SortInsertionSort(A) for j = 2 to A.length key=[Aj] i=j-1 while i > 0 and A[i] > key A[i+1] = A[i] i=i-1 A[i+1] = key 9
10. 10. That Makes no Sense, Show me! 10
11. 11. So What’s The Running Time? 11
12. 12. Sum Running Time for Each Statement...T(n) = c1n+c2(n-1)+c3(n-1)+c4 sum(tj) j=2..n+c5 sum(tj-1) j=2..n+c6sum(tj-1) j=2..n+c7(n-1) 12
13. 13. Best Case Running TimeIf the input (A) is already sorted then...A[i] <= key when has initial value of j-1 thus tj=1.And so...T(n) = c1n+c2(n-1)+c3(n-1)+c4(n-1)+c7(n-1)= (c1+c2+c3+c4+c7)n-(c2+c3+c4+c7)Which can be expressed as an+b for constants a and b that depend on ciSo T(n) is a linear function of n 13
14. 14. 14
15. 15. Side Note: No One Cares About Best Case 15
16. 16. Worst Case ScenarioIf the input (n) is in reverse sort order then...We have to compare each A[j] with each element in the sub array A[1..j-1].And so...T(n) = (c4/2+c5/2+c6/2)n^2 +(c1 +c2+c3+c4/2- c5/2-c6/2+c7)n-(c2+c3+c4+c7)Which can be expressed as an^2 + bn + cSo T(n) is a quadratic function of n 16
17. 17. 17
18. 18. In Short...In worst case insertion sort sucks!  18
19. 19. Man That Was a Lot of Maths! 19
20. 20. Simplifying Things With Asymptotic Notation• Asymptotic notation characterises functions by their growth rates• Functions with the same growth rates have the same Asymptotic notation 20
21. 21. How Does That Help Us?Let’s say we have a function with running timeT(n) = 4n^2 - 2n + 2If n = 500 then4n^2 is 1000 times bigger than 2nSo...We can ignore smaller order terms and coefficientsT(n) = 4n^2 -2n +2 can be written O(n) = n^2 21
22. 22. A Short Note on The Abuse of “=“If T(n) = 4n^2 -2n +2Then saying T(n) = O(n^2) is not strictly correctRather T(n) is in the set O(n^2) and the above should be read as T(n) is O(n^2) and not T(n) equals O(n^2)But really on Maths geeks care – Ed.  22
23. 23. So Back to Insertion SortSo now we can say of Insertion Sort that...Best case it’s O(n)And worst case it’s O(n^2)And since we only care about worst case...We say that Insertion Sort has O(n^2)Which sucks! – Ed.  23
24. 24. Designing Algorithms So can we do better? 24
25. 25. Optimizing Algorithms is Child’s Play• Sit at table• Foreach item in itemsOnPlate – Eat item• Wait(MealComplete)• Foreach dish in dishesUsed – WashDish – DryDish• Resume Play 25
26. 26. Child Will Optimize To…• Pause Game• Set Speed = MaxInt• Run to table• Take sliceBread(1)• Foreach item on Plate – Place item on bread• Take sliceBread(2)• Run Outside• Resume Game 26
27. 27. Divide And Conquer• Divide – Divide the problem into sub problems• Conquer – Solve the sub problems recursively• Combine – Add the solutions to the sub problems into the solution for the original problem. 27
28. 28. Merge Sort• Divide – Divide the n elements into two n/2 element arrays• Conquer – Sort the two arrays recursively• Combine – Merge the two sorted arrays to produce the answer. 28
29. 29. Analysis of Merge SortMergeSort(A,p,r) if(p<r) q = [(p+r)/2] MergeSort(A,p,q) MergeSort(A,q+1,r) Merge(A,p,q,r)Initial call MergeSort(A,1,A.length) 29
30. 30. Dancers, or it Never Happened!! 30
31. 31. So What’s The Running Time?In the general case...If the divide step yields ‘a’ sub problemsEach 1/b the size of the originalIt takes T(n/b) time to solve one problem of n/b sizeSo it takes aT(n/b) to solve ‘a’ of themThen, if it takes D(n) time to divide the problemAnd C(n) time to combine the resultsThen we get the recurrence...T(n) = aT(n/b) + D(n) + C(n). 31
32. 32. Apply That to Merge Sort...• Divide – Computes the middle of the subarray, taking constant time so, D(n) = O(1)• Conquer – Recursively solve two sub problems each of size n/2 contributing 2T(n/2) to the running time• Combine – Merge procedure O(n)• Giving us a recurrence of 2T(n/2)+O(n) 32
33. 33. Solve The Recurrence Using The Master MethodFor a Recurrence in the formT(n) = aT(n/b) + f(n)ThenIf f(n) = O(nlogba-k) then T(n) = O(nlogba)If f(n) = O(nlogba) then T(n) = O(nlogba log n)if f(n) = Omega(n log b a+k) and if af(n/b) <= cf(n) then T(n) = O(f(n)) 33
34. 34. What?!• More simply we are comparing f(n) with the function n log ba and intuitively understanding that the bigger of the two determines the solution to the recurrence. 34
35. 35. And So...• With Merge Sort we are in the third case of the Master Method thus...• T(n) = O(n log n)• Which is much better than the O(n^2) of Insertion Sort 35
36. 36. 36
37. 37. What We Learned• Performance is important• Therefore algorithmic optimization is too• We have a model to benchmark• And a syntax• Divide and conquer• Master Method• Other resources. 37
38. 38. 38
39. 39. Questions? 39