Heapsort quick sort
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Heapsort quick sort

on

  • 463 views

Heapsort

Heapsort
quick sort

Statistics

Views

Total Views
463
Views on SlideShare
463
Embed Views
0

Actions

Likes
0
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Heapsort quick sort Presentation Transcript

  • 1. Algorithms Sandeep Kumar Poonia Head Of Dept. CS/IT B.E., M.Tech., UGC-NET LM-IAENG, LM-IACSIT,LM-CSTA, LM-AIRCC, LM-SCIEI, AM-UACEE
  • 2. Algorithms Introduction to heapsort Quicksort Sandeep Kumar Poonia
  • 3. Recap Asymptotic notation Merge Sort Solving Recurrences The Master Theorem Sandeep Kumar Poonia
  • 4. Sorting Revisited  So far we’ve talked about two algorithms to sort an array of numbers    What is the advantage of merge sort? What is the advantage of insertion sort? Next on the agenda: Heapsort    Combines advantages of both previous algorithms Sort in Place – Like insertion sort O(n Lg n) Worst case – Like Merge sort Sandeep Kumar Poonia
  • 5. Heaps  A heap can be seen as a complete binary tree: 16 14 10 8 2   7 4 9 1 What makes a binary tree complete? Is the example above complete? Sandeep Kumar Poonia 3
  • 6. Heaps  A heap can be seen as a complete binary tree: 16 14 10 8 2  7 4 1 9 1 1 3 1 1 We calls them “nearly complete” binary trees; can think of unfilled slots as null pointers Sandeep Kumar Poonia 1
  • 7. Heaps  In practice, heaps are usually implemented as arrays: 16 14 A = 16 14 10 8 7 9 3 2 4 8 1 = 2 Sandeep Kumar Poonia 10 7 4 1 9 3
  • 8. Heaps  To represent a complete binary tree as an array:      The root node is A[1] Node i is A[i] The parent of node i is A[i/2] (note: integer divide) The left child of node i is A[2i] The right child of node i is A[2i + 1] 16 14 A = 16 14 10 8 7 9 3 2 4 8 1 = 2 Sandeep Kumar Poonia 10 7 4 1 9 3
  • 9. Referencing Heap Elements  So… Parent(i) { return i/2; } Left(i) { return 2*i; } right(i) { return 2*i + 1; }  An aside: How would you implement this most efficiently? Sandeep Kumar Poonia
  • 10. The Heap Property  Heaps also satisfy the heap property: A[Parent(i)]  A[i] for all nodes i > 1    In other words, the value of a node is at most the value of its parent Where is the largest element in a heap stored? Definitions:   The height of a node in the tree = the number of edges on the longest downward path to a leaf The height of a tree = the height of its root Sandeep Kumar Poonia
  • 11. Heap Height Q: What are the minimum and maximum numbers of elements in a heap of height h? Ans: Since a heap is an almost-complete binary tree (complete at all levels except possibly the lowest), it has at most 2h+1 -1 elements (if it is complete) and at least 2h -1+1 = 2h elements (if the lowest level has just 1 element and the other levels are complete). Sandeep Kumar Poonia
  • 12. Heap Height Q: What is the height of an n-element heap? Why? Ans: This is nice: basic heap operations take at most time proportional to the height of the heap  Given an n-element heap of height h, we know that  Thus  Since h is an integer, Sandeep Kumar Poonia
  • 13. Heap Operations: Heapify()  Heapify(): maintain the heap property     Given: a node i in the heap with children l and r Given: two subtrees rooted at l and r, assumed to be heaps Problem: The subtree rooted at i may violate the heap property. Action: let the value of the parent node “float down” so subtree at i satisfies the heap property  What do you suppose will be the basic operation between i, l, and r? Sandeep Kumar Poonia
  • 14. Procedure MaxHeapify MaxHeapify(A, i) 1. l  left(i) 2. r  right(i) 3. if l  heap-size[A] and A[l] > A[i] 4. then largest  l 5. else largest  i 6. if r  heap-size[A] and A[r] > A[largest] 7. then largest  r 8. if largest i 9. then exchange A[i]  A[largest] 10. MaxHeapify(A, largest) Sandeep Kumar Poonia Assumption: Left(i) and Right(i) are max-heaps.
  • 15. Running Time for MaxHeapify MaxHeapify(A, i) 1. l  left(i) 2. r  right(i) 3. if l  heap-size[A] and A[l] > A[i] 4. then largest  l 5. else largest  i 6. if r  heap-size[A] and A[r] > A[largest] 7. then largest  r 8. if largest i 9. then exchange A[i]  A[largest] 10. MaxHeapify(A, largest) Sandeep Kumar Poonia Time to fix node i and its children = (1) PLUS Time to fix the subtree rooted at one of i’s children = T(size of subree at largest)
  • 16. Running Time for MaxHeapify(A, n)  T(n) = T(largest) + (1)  largest  2n/3 (worst case occurs when the last row of tree is exactly half full)  T(n)  T(2n/3) + (1)  T(n) = O(lg n)  Alternately, MaxHeapify takes O(h) where h is the height of the node where MaxHeapify is applied Sandeep Kumar Poonia
  • 17. Building a heap  Use MaxHeapify to convert an array A into a max-heap.  Call MaxHeapify on each element in a bottom-up manner. BuildMaxHeap(A) 1. heap-size[A]  length[A] 2. for i  length[A]/2 downto 1 3. do MaxHeapify(A, i) Sandeep Kumar Poonia
  • 18. BuildMaxHeap – Example Input Array: 24 21 23 22 36 29 30 34 28 27 Initial Heap: (not max-heap) 24 21 36 22 34 Sandeep Kumar Poonia 23 28 27 29 30
  • 19. BuildMaxHeap – Example MaxHeapify(10/2 = 5) MaxHeapify(4) MaxHeapify(3) 24 36 21 34 24 36 30 23 MaxHeapify(2) MaxHeapify(1) 36 27 21 28 34 24 22 34 22 Sandeep Kumar Poonia 24 28 27 21 29 30 23
  • 20. Correctness of BuildMaxHeap   Loop Invariant: At the start of each iteration of the for loop, each node i+1, i+2, …, n is the root of a max-heap. Initialization:    Before first iteration i = n/2 Nodes n/2+1, n/2+2, …, n are leaves and hence roots of max-heaps. Maintenance:    By LI, subtrees at children of node i are max heaps. Hence, MaxHeapify(i) renders node i a max heap root (while preserving the max heap root property of higher-numbered nodes). Decrementing i reestablishes the loop invariant for the next iteration. Sandeep Kumar Poonia
  • 21. Running Time of BuildMaxHeap  Loose upper bound:    Cost of a MaxHeapify call  No. of calls to MaxHeapify O(lg n)  O(n) = O(nlg n) Tighter bound:     Cost of a call to MaxHeapify at a node depends on the height, h, of the node – O(h). Height of most nodes smaller than n. Height of nodes h ranges from 0 to lg n. No. of nodes of height h is n/2h+1 Sandeep Kumar Poonia
  • 22. Heapsort   Sort by maintaining the as yet unsorted elements as a max-heap. Start by building a max-heap on all elements in A.   Move the maximum element to its correct final position.   Decrement heap-size[A]. Restore the max-heap property on A[1..n–1].   Exchange A[1] with A[n]. Discard A[n] – it is now sorted.   Maximum element is in the root, A[1]. Call MaxHeapify(A, 1). Repeat until heap-size[A] is reduced to 2. Sandeep Kumar Poonia
  • 23. Heapsort(A) HeapSort(A) 1. Build-Max-Heap(A) 2. for i  length[A] downto 2 3. do exchange A[1]  A[i] 4. heap-size[A]  heap-size[A] – 1 5. MaxHeapify(A, 1) Sandeep Kumar Poonia
  • 24. Heapify() Example 16 4 10 14 2 7 8 3 1 A = 16 4 10 14 7 Sandeep Kumar Poonia 9 9 3 2 8 1
  • 25. Heapify() Example 16 4 10 14 2 7 8 3 1 A = 16 4 10 14 7 Sandeep Kumar Poonia 9 9 3 2 8 1
  • 26. Heapify() Example 16 4 10 14 2 7 8 3 1 A = 16 4 10 14 7 Sandeep Kumar Poonia 9 9 3 2 8 1
  • 27. Heapify() Example 16 14 10 4 2 7 8 3 1 A = 16 14 10 4 Sandeep Kumar Poonia 9 7 9 3 2 8 1
  • 28. Heapify() Example 16 14 10 4 2 7 8 3 1 A = 16 14 10 4 Sandeep Kumar Poonia 9 7 9 3 2 8 1
  • 29. Heapify() Example 16 14 10 4 2 7 8 3 1 A = 16 14 10 4 Sandeep Kumar Poonia 9 7 9 3 2 8 1
  • 30. Heapify() Example 16 14 10 8 2 7 4 3 1 A = 16 14 10 8 Sandeep Kumar Poonia 9 7 9 3 2 4 1
  • 31. Heapify() Example 16 14 10 8 2 7 4 3 1 A = 16 14 10 8 Sandeep Kumar Poonia 9 7 9 3 2 4 1
  • 32. Heapify() Example 16 14 10 8 2 7 4 3 1 A = 16 14 10 8 Sandeep Kumar Poonia 9 7 9 3 2 4 1
  • 33. Algorithm Analysis HeapSort(A) 1. Build-Max-Heap(A) 2. for i  length[A] downto 2 3. do exchange A[1]  A[i] 4. heap-size[A]  heap-size[A] – 1 5. MaxHeapify(A, 1)  In-place  Not Stable  Build-Max-Heap takes O(n) and each of the n-1 calls to Max-Heapify takes time O(lg n).  Therefore, T(n) = O(n lg n) Sandeep Kumar Poonia
  • 34. Heap Procedures for Sorting MaxHeapify  BuildMaxHeap  HeapSort  Sandeep Kumar Poonia O(lg n) O(n) O(n lg n)
  • 35. Priority Queue       Popular & important application of heaps. Max and min priority queues. Maintains a dynamic set S of elements. Each set element has a key – an associated value. Goal is to support insertion and extraction efficiently. Applications:   Ready list of processes in operating systems by their priorities – the list is highly dynamic In event-driven simulators to maintain the list of events to be simulated in order of their time of occurrence. Sandeep Kumar Poonia
  • 36. Basic Operations  Operations on a max-priority queue:  Insert(S, x) - inserts the element x into the set S       S  S  {x}. Maximum(S) - returns the element of S with the largest key. Extract-Max(S) - removes and returns the element of S with the largest key. Increase-Key(S, x, k) – increases the value of element x’s key to the new value k. Min-priority queue supports Insert, Minimum, Extract-Min, and Decrease-Key. Heap gives a good compromise between fast insertion but slow extraction and vice versa. Sandeep Kumar Poonia
  • 37. Heap Property (Max and Min)  Max-Heap  For every node excluding the root, value is at most that of its parent: A[parent[i]]  A[i]  Largest element is stored at the root. In any subtree, no values are larger than the value stored at subtree root.  Min-Heap     For every node excluding the root, value is at least that of its parent: A[parent[i]]  A[i] Smallest element is stored at the root. In any subtree, no values are smaller than the value stored at subtree root Sandeep Kumar Poonia
  • 38. Heap-Extract-Max(A) Implements the Extract-Max operation. Heap-Extract-Max(A,n) 1. if n < 1 2. then error “heap underflow” 3. max  A[1] 4. A[1]  A[n] 5. n  n - 1 6. MaxHeapify(A, 1) 7. return max Running time : Dominated by the running time of MaxHeapify = O(lg n) Sandeep Kumar Poonia
  • 39. Heap-Insert(A, key) Heap-Insert(A, key) 1. heap-size[A]  heap-size[A] + 1 2. i  heap-size[A] 4. while i > 1 and A[Parent(i)] < key 5. do A[i]  A[Parent(i)] 6. i  Parent(i) 7. A[i]  key Running time is O(lg n) The path traced from the new leaf to the root has length O(lg n) Sandeep Kumar Poonia
  • 40. Heap-Increase-Key(A, i, key) Heap-Increase-Key(A, i, key) 1 If key < A[i] 2 then error “new key is smaller than the current key” 3 A[i]  key 4 while i > 1 and A[Parent[i]] < A[i] 5 do exchange A[i]  A[Parent[i]] 6 i  Parent[i] Heap-Insert(A, key) 1 heap-size[A]  heap-size[A] + 1 2 A[heap-size[A]]  – 3 Heap-Increase-Key(A, heap-size[A], key) Sandeep Kumar Poonia
  • 41. Quicksort Sorts in place  Sorts O(n lg n) in the average case  Sorts O(n2) in the worst case  So why would people use it instead of merge sort?  Sandeep Kumar Poonia
  • 42. Quicksort  Another divide-and-conquer algorithm  The array A[p..r] is partitioned into two nonempty subarrays A[p..q] and A[q+1..r]  Invariant: All elements in A[p..q] are less than all elements in A[q+1..r]   The subarrays are recursively sorted by calls to quicksort Unlike merge sort, no combining step: two subarrays form an already-sorted array Sandeep Kumar Poonia
  • 43. Quicksort Code Quicksort(A, p, r) { if (p < r) { q = Partition(A, p, r); Quicksort(A, p, q-1); Quicksort(A, q+1, r); } } Sandeep Kumar Poonia
  • 44. Partition  Clearly, all the action takes place in the partition() function   Rearranges the subarray in place End result:  Two subarrays  All values in first subarray  all values in second   Returns the index of the “pivot” element separating the two subarrays How do you suppose we implement this function? Sandeep Kumar Poonia
  • 45. Partition In Words  Partition(A, p, r):   Select an element to act as the “pivot” (which?) Grow two regions, A[p..i] and A[j..r]  All elements in A[p..i] <= pivot  All elements in A[j..r] >= pivot      Increment i until A[i] >= pivot Decrement j until A[j] <= pivot Swap A[i] and A[j] Repeat until i >= j Return j Sandeep Kumar Poonia
  • 46. Partition Code What is the running time of partition()? Sandeep Kumar Poonia
  • 47. Review: Analyzing Quicksort  What will be the worst case for the algorithm?   What will be the best case for the algorithm?   Partition is balanced Which is more likely?   Partition is always unbalanced The latter, by far, except... Will any particular input elicit the worst case?  Yes: Already-sorted input Sandeep Kumar Poonia
  • 48. Review: Analyzing Quicksort  In the worst case: one subarray have 0 element and another n-1 elements T(1) = (1) T(n) = T(n - 1) + (n)  Works out to T(n) = (n2) Sandeep Kumar Poonia
  • 49. Review: Analyzing Quicksort  In the best case: each has <= n/2 elements T(n) = 2T(n/2) + (n)  What does this work out to? T(n) = (n lg n) Sandeep Kumar Poonia
  • 50. Review: Analyzing Quicksort (Average Case)  Intuitively, a real-life run of quicksort will produce a mix of “bad” and “good” splits    Randomly distributed among the recursion tree Pretend for intuition that they alternate between best-case (n/2 : n/2) and worst-case (n-1 : 1) What happens if we bad-split root node, then good-split the resulting size (n-1) node? Sandeep Kumar Poonia
  • 51. Review: Analyzing Quicksort (Average Case)  Intuitively, a real-life run of quicksort will produce a mix of “bad” and “good” splits    Randomly distributed among the recursion tree Pretend for intuition that they alternate between best-case (n/2 : n/2) and worst-case (n-1 : 1) What happens if we bad-split root node, then good-split the resulting size (n-1) node?  We end up with three subarrays, size 1, (n-1)/2, (n-1)/2  Combined cost of splits = n + n -1 = 2n -1 = O(n)  No worse than if we had good-split the root node! Sandeep Kumar Poonia
  • 52. Review: Analyzing Quicksort (Average Case) Intuitively, the O(n) cost of a bad split (or 2 or 3 bad splits) can be absorbed into the O(n) cost of each good split  Thus running time of alternating bad and good splits is still O(n lg n), with slightly higher constants  Sandeep Kumar Poonia
  • 53. Analyzing Quicksort: Average Case  For simplicity, assume:   All inputs distinct (no repeats) Slightly different partition() procedure  partition around a random element, which is not included in subarrays  all splits (0:n-1, 1:n-2, 2:n-3, … , n-1:0) equally likely What is the probability of a particular split happening?  Answer: 1/n  Sandeep Kumar Poonia
  • 54. Analyzing Quicksort: Average Case So partition generates splits (0:n-1, 1:n-2, 2:n-3, … , n-2:1, n-1:0) each with probability 1/n  If T(n) is the expected running time,  1 n1 T n    T k   T n  1  k   n  n k 0 What is each term under the summation for?  What is the (n) term for?  Sandeep Kumar Poonia
  • 55. Analyzing Quicksort: Average Case  So… 1 n 1 T n    T k   T n  1  k   n  n k 0 2 n 1   T k   n  n k 0 Sandeep Kumar Poonia Write it on the board
  • 56. Analyzing Quicksort: Average Case  We can solve this recurrence using the substitution method     Guess the answer Assume that the inductive hypothesis holds Substitute it in for some value < n Prove that it follows for n Sandeep Kumar Poonia
  • 57. Analyzing Quicksort: Average Case  We can solve this recurrence using the substitution method  Guess the answer  What’s    the answer? Assume that the inductive hypothesis holds Substitute it in for some value < n Prove that it follows for n Sandeep Kumar Poonia
  • 58. Analyzing Quicksort: Average Case  We can solve this recurrence using the dreaded substitution method  Guess the answer  T(n)    = O(n lg n) Assume that the inductive hypothesis holds Substitute it in for some value < n Prove that it follows for n Sandeep Kumar Poonia
  • 59. Analyzing Quicksort: Average Case  We can solve this recurrence using the dreaded substitution method  Guess the answer  T(n)  = O(n lg n) Assume that the inductive hypothesis holds  What’s   the inductive hypothesis? Substitute it in for some value < n Prove that it follows for n Sandeep Kumar Poonia
  • 60. Analyzing Quicksort: Average Case  We can solve this recurrence using the dreaded substitution method  Guess the answer  T(n)  Assume that the inductive hypothesis holds  T(n)   = O(n lg n)  an lg n + b for some constants a and b Substitute it in for some value < n Prove that it follows for n Sandeep Kumar Poonia
  • 61. Analyzing Quicksort: Average Case  We can solve this recurrence using the dreaded substitution method  Guess the answer  T(n)  Assume that the inductive hypothesis holds  T(n)  = O(n lg n)  an lg n + b for some constants a and b Substitute it in for some value < n  What  value? Prove that it follows for n Sandeep Kumar Poonia
  • 62. Analyzing Quicksort: Average Case  We can solve this recurrence using the dreaded substitution method  Guess the answer  T(n)  Assume that the inductive hypothesis holds  T(n)   an lg n + b for some constants a and b Substitute it in for some value < n  The  = O(n lg n) value k in the recurrence Prove that it follows for n Sandeep Kumar Poonia
  • 63. Analyzing Quicksort: Average Case  We can solve this recurrence using the dreaded substitution method  Guess the answer  T(n)  Assume that the inductive hypothesis holds  T(n)   an lg n + b for some constants a and b Substitute it in for some value < n  The  = O(n lg n) value k in the recurrence Prove that it follows for n  Grind Sandeep Kumar Poonia through it…
  • 64. Analyzing Quicksort: Average Case 2 n 1 T n    T k   n  n k 0 2 n 1   ak lg k  b   n  n k 0 The recurrence to be solved Plug in inductive hypothesis What are we doing here? 2  n 1  What are we doing here?  b   ak lg k  b   n  Expand out the k=0 case n  k 1  2 n 1 2b   ak lg k  b    n  n k 1 n 2b/n is just a constant, What are we doing here? so fold it into (n) 2 n 1   ak lg k  b   n  n k 1 Note: leaving the same recurrence as the book Sandeep Kumar Poonia
  • 65. Analyzing Quicksort: Average Case 2 n 1 T n    ak lg k  b   n  n k 1 2 n 1 2 n 1   ak lg k   b  n  n k 1 n k 1 The recurrence to be solved Distribute the summation What are we doing here? 2a n 1 2b the summation:  k lg k  (n  1)  n  Evaluateare we doing here? What  b+b+…+b = b (n-1) n k 1 n 2a n 1   k lg k  2b  n  n k 1 This summation gets its own set of slides later Sandeep Kumar Poonia Since n-1<n,we doing here? What are 2b(n-1)/n < 2b
  • 66. Analyzing Quicksort: Average Case 2a n 1 T n    k lg k  2b  n  n k 1     The recurrence to be solved 2a  1 2 1 2 We’ll prove this  n lg n  n   2b  n  What the hell? later n 2 8  a an lg n  n  2b  n  Distribute the (2a/n) term What are we doing here? 4 a  Remember, our goal is to get  an lg n  b   n   b  n  What are we doing here? T(n)  an lg n + b 4   Pick a large enough that an lg n  b How did we do this? an/4 dominates (n)+b Sandeep Kumar Poonia
  • 67. Analyzing Quicksort: Average Case  So T(n)  an lg n + b for certain a and b    Thus the induction holds Thus T(n) = O(n lg n) Thus quicksort runs in O(n lg n) time on average Sandeep Kumar Poonia