Upcoming SlideShare
×

# Binomial Heaps and Fibonacci Heaps

• 1,975 views

Heaps and Priority Queues allow data to be accessed in an order. Binary heaps are great, but don't support merging (unions). Binomial heaps solve that problem. Dijkstra and Prim's algorithm can …

Heaps and Priority Queues allow data to be accessed in an order. Binary heaps are great, but don't support merging (unions). Binomial heaps solve that problem. Dijkstra and Prim's algorithm can benefit greatly from using a decrease key operation that runs in O(1) time. Fibonacci heaps provide that, while keeping the extract min operation to O(log n) time. Amortized analysis can be used for both.

• Comment goes here.
Are you sure you want to
Be the first to comment

Total Views
1,975
On Slideshare
0
From Embeds
0
Number of Embeds
4

Shares
0
0
Likes
3

No embeds

### Report content

No notes for slide

### Transcript

• 1. Advanced Data Structures PRIORITY QUEUES – BINARY HEAPS, BINOMIAL HEAPS FIBONACCI HEAPS
• 2.  Instructor  Prof. Amrinder Arora  Contact me: http://www.standardwisdom.com/contact-me/  Other Links  Software Blog: http://sj.standardwisdom.com  Algorithms Textbook: http://www.standardwisdom.com/algorithms-book/ LOGISTICS Heaps and Priority Queues Advanced Data Structures - Arora 2
• 3. Advanced Data Structures Basics Record / Struct Arrays / Linked Lists / Stacks / Queues Graphs / Trees / BSTs Advanced Trie, B-Tree Splay Trees R-Trees Heaps and PQs Union Find, Sets WHERE WE ARE Heaps and Priority Queues Advanced Data Structures - Arora 3
• 4.  Robert Tarjan – Princeton  Rada Mihalcea – UNT  Dan Melamed – NYU  Ata Türk – Bilkent Univ. CREDITS Heaps and Priority Queues Advanced Data Structures - Arora 4
• 5.  Ice cream costs vary by flavor: \$3 (vanilla), \$4 (strawberry), or \$5 (chocolate). All flavors of ice cream are equally likely.  Average cost = \$4.  Ice cream costs \$10. Every 10th ice cream additionally gives you a \$60 voucher (redeemable in cash).  Amortized cost = \$4. BUT FIRST, AN ICE CREAM Heaps and Priority Queues Advanced Data Structures - Arora 5
• 6.  Ice cream costs \$1  Every 10th ice cream has a surcharge of \$10.  Amortized cost = \$2 AMORTIZED VS. AVERAGE (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 6
• 7.  A binary tree T that satisfies two properties:  MinHeap: key(parent)  key(child)  [OR MaxHeap: key(parent)  key(child)]  All levels are full, except the last one, which is left-filled HEAPS – BINARY TREES Heaps and Priority Queues Advanced Data Structures - Arora 7
• 8.  To implement priority queues  Priority queue = a queue where all elements have a “priority” associated with them  Remove in a priority queue removes the element with the smallest priority  insert  removeMin WHAT ARE HEAPS USEFUL FOR? Heaps and Priority Queues Advanced Data Structures - Arora 8
• 9.  A heap T storing n keys has height h = log(n), which is O(log n). [The statement is true for almost complete binary trees in general.] HEAP PROPERTIES Heaps and Priority Queues Advanced Data Structures - Arora 9
• 10.  Using arrays.  If indexed from 1: Parent = k ; Children = 2k , 2k+1  Efficient! [No pointers. Just array indexes.] HEAP IMPLEMENTATION [4] 6 12 7 1918 9 6 9 7 10 30 31 [1] [2] [3] [5] [6] [1] [2] [3] [4] [1] [2] Heaps and Priority Queues Advanced Data Structures - Arora 10
• 11.  Insert 3 HEAP INSERTION Heaps and Priority Queues Advanced Data Structures - Arora 11
• 12.  Just insert it at the first empty slot.  Upheap (Shift the key up), if necessary HEAP INSERTION (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 12
• 13.  Continue the upheap process, until:  Either the key is smaller than the parent,  Or it becomes the root. [We have a new minimum] HEAP INSERTION (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 13
• 14.  Remove element from priority queues – removeMin() or extractMin()  Remove the root, replace with the last element.  “Downheap” (Swap the node with the smaller of the child nodes) if necessary HEAP REMOVAL Heaps and Priority Queues Advanced Data Structures - Arora 14
• 15.  Terminate downheap when  reach leaf level  key parent is greater than key child HEAP REMOVAL (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 15
• 16.  One can build a heap simply by doing n inserts.  This takes O(n log n) time. (Why?)  Instead, we can build a heap faster, in O(n) time  Build it from ground up BUILDING A HEAP Heaps and Priority Queues Advanced Data Structures - Arora 16
• 17.  Step 1: Build a heap: O(n) time  Step 2: removeMin() n times  Running time: O(n log n)  Heap sort is generally quite efficient, and competes effectively against quick sort. (Both heap sort and quick sort are significantly faster than merge sort, although of course, not in asymptotic terms.) HEAP SORTING Heaps and Priority Queues Advanced Data Structures - Arora 17
• 18. Operation Binary Heap Tree Insert O(log n) Extract Minimum O(log n) Find Minimum O(1) HEAPS, SUMMARY It is not easy to merge two binary heaps, especially if they are of different sizes. If we decrease the key of a node, upheap can be done in O(log n) time. Heaps and Priority Queues Advanced Data Structures - Arora 18
• 19. Union, Decrease Key, Random Delete MORE ACTIONS ON HEAPS Heaps and Priority Queues Advanced Data Structures - Arora 19
• 20.  MAKE-HEAP ()  Creates & returns a new heap with no elements.  INSERT (H,x)  Inserts a node x whose key field has already been filled into heap H.  MINIMUM (H)  Returns a pointer to the node in heap H whose key is minimum.  EXTRACT-MIN (H)  Deletes the node from heap H whose key is minimum.  DECREASE-KEY (H, x, k)  Decreases the key value of node x within heap H the new value k  DELETE (H, x)  Deletes node x from heap H.  UNION (H1, H2)  Creates and returns a new heap that contains all nodes of heaps H1 & H2.  Heaps H1 & H2 are destroyed by this operation MERGEABLE HEAPS As before New Oper- ations Heaps and Priority Queues Advanced Data Structures - Arora 20
• 21.  A binomial heap is a collection of binomial trees.  The binomial tree Bk is an ordered tree defined recursively  B0 – Consists of a single node  Bk – Consists of two binominal trees Bk-1 linked together. Root of one is the leftmost child of the root of the other. BINOMIAL TREES Heaps and Priority Queues Advanced Data Structures - Arora 21
• 22. BINOMIAL TREES B k-1 B k-1 B k Bo Heaps and Priority Queues Advanced Data Structures - Arora 22
• 23. BINOMIAL TREES B4 B0 B1 B2 B 1 B1 B3 B2 B1 B0 B3 B2 B1 B0 Heaps and Priority Queues Advanced Data Structures - Arora 23
• 24. BINOMIAL TREES Bk Bk-2 Bk-1 B2 B1 Bo Heaps and Priority Queues Advanced Data Structures - Arora 24
• 25. For the binomial tree Bk 1. There are 2k nodes, 2. The height of tree is k, 3. There are exactly C(k,i) nodes at depth i for i= 0,1,..,k and 4. The root has the highest degree, k. This is higher than the degree of any other node 5. Children of the root are the roots of subtrees B0, B1 B2, B3 … Bk-1 PROPERTIES OF BINOMIAL TREES Heaps and Priority Queues Advanced Data Structures - Arora 25
• 26.  PROOF: By induction on k PROPERTIES OF BINOMIAL TREES (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 26
• 27.  COROLLARY: The maximum degree of any node in an n-node binomial tree is lg(n)  The term BINOMIAL TREE comes from the 3rd property. C(k,i) is the binomial coefficient. PROPERTIES OF BINOMIAL TREES (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 27
• 28.  A BINOMIAL HEAP H is a set of BINOMIAL TREES that satisfies the following “Binomial Heap Properties”  Each binomial tree in H is HEAP-ORDERED  The key of a node is ≥ the key of the parent  Root of each binomial tree in H contains the smallest key in that tree. BINOMIAL HEAPS Heaps and Priority Queues Advanced Data Structures - Arora 28
• 29. In a consolidated binomial heap, there is at most one binomial tree in H whose root has a given degree.  n-node binomial heap H consists of at most [lgn] + 1 binomial trees. (Binary represantation of n has lg(n) + 1 bits) BINOMIAL HEAPS (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 29
• 30.  Example: A binomial heap with n = 13 nodes  3 2 1 0  13 =< 1, 1, 0, 1>2  Consists of B0, B2, B3  head[H] BINOMIAL HEAPS 10 25 1 12 18 6 298 38 14 27 1711 B0 B2 B3 Heaps and Priority Queues Advanced Data Structures - Arora 30
• 31.  Each binomial tree within a binomial heap is stored in the left-child, right-sibling representation  Each node X contains POINTERS  p[x] to its parent  child[x] to its leftmost child  sibling[x] to its immediately right sibling  Each node X also contains the field degree[x] which denotes the number of children of X. REPRESENTATION OF BINOMIAL HEAPS Heaps and Priority Queues Advanced Data Structures - Arora 31
• 32. REPRESENTATION OF BINOMIAL HEAPS HEAD [H] 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 parent degree key child sibling ROOT LIST (LINKED LIST) Heaps and Priority Queues Advanced Data Structures - Arora 32
• 33.  Lazy Approach – Consolidate only when needed (data structure can break some properties at times)  Eager Approach – Consolidate often (keep the data structure clean at all times) CONSOLIDATION – LAZY VS. EAGER Heaps and Priority Queues Advanced Data Structures - Arora 33
• 34.  Lazy Approach  Insert – Simply add to the list and update minimum if needed  Union – Simply combine the list using pointers, and update minimum if needed  Delete – Delete the minimum, merge the lists and consolidate  Eager Approach  Insert – Create a new heap, union and consolidate  Union – Combine the lists and consolidate  Delete – Delete the minimum, merge the lists and consolidate CONSOLIDATION, IN CONTEXT OF BINOMIAL HEAPS Heaps and Priority Queues Advanced Data Structures - Arora 34
• 35.  Insert – O(log n) time to consolidate the lists  Union – O(log n) time to consolidate the lists  Delete – O(log n) time to consolidate the lists BINOMIAL HEAP (EAGER APPROACH) OPERATIONS Heaps and Priority Queues Advanced Data Structures - Arora 35
• 36.  Use a potential function: f(H) = trees(H), which is simply the number of trees in H.  Insert  Actual cost: O(1)  Change in potential function = 1.  Amortized cost: O(1)  Union  Actual cost: O(1)  Change in potential function = 0.  Amortized cost: O(1)  Delete  Actual cost = O(number of trees in the heap + max degree) = # of trees + O(log n)  Change in potential function = O(log n) - # trees  Amortized cost: O(log n) + O(max degree) = O(log n) BINOMIAL HEAP (LAZY APPROACH) - OPERATIONS Heaps and Priority Queues Advanced Data Structures - Arora 36
• 37. Dijkstra and Prim DECREASE KEY – A CENTRAL PROBLEM Heaps and Priority Queues Advanced Data Structures - Arora 37
• 38.  Decrease key operation appears frequently in two common algorithms:  Dijkstra’s algorithm for Shortest Path  Prim’s algorithm for Minimum Spanning Tree  The data structures studied so far either do not support the decrease key operation, or can be made to support it in O(log n) time.  Fibonacci heaps provide decrease key operation in O(1) amortized time.  This changes the time of those two algorithms from O(m log n) to O(m + n log n), where m is the number of edges and n is the number of vertices in the graph. DECREASE KEY Heaps and Priority Queues Advanced Data Structures - Arora 38
• 39.  History. [Fredman and Tarjan, 1986]  Ingenious data structure and analysis.  Original motivation: improve Dijkstra's shortest path algorithm from O(E log V ) to O(E + V log V ).  Basic idea.  Similar to binomial heaps, but less rigid structure.  Uses the concept of a “mark” to support decrease key operation  Historical perspective/question: While now a days we study both lazy and eager versions of binomial heaps, it is this researcher’s (Arora’s) belief that binomial heaps essentially meant eager version before the appearance of Fibonacci heaps. FIBONACCI HEAPS Heaps and Priority Queues Advanced Data Structures - Arora 39
• 40.  Fibonacci heap.  Very similar to Binomial heap, it is a linked list of heap-ordered trees.  Maintain pointer to minimum element.  Set of “marked” nodes (To be explained shortly) FIBONACCI HEAPS: STRUCTURE 723 30 17 35 26 46 24 Heap H 39 4118 52 3 44 roots heap-ordered tree Heaps and Priority Queues Advanced Data Structures - Arora 40
• 41.  Notation.  n = number of nodes in heap.  rank(x) = number of children of node x.  rank(H) = max rank of any node in heap H.  trees(H) = number of trees in heap H.  marks(H) = number of marked nodes in heap H. FIBONACCI HEAPS: NOTATION 723 30 17 35 26 46 24 39 4118 52 3 44 rank = 3 min Heap H trees(H) = 5 marks(H) = 3 marked n = 14 Heaps and Priority Queues Advanced Data Structures - Arora 41
• 42.  We will use amortized analysis using potential function technique  Potential of heap H, (H) = trees(H) + 2  marks(H) FIBONACCI HEAPS: POTENTIAL FUNCTION 723 30 17 35 26 46 24 (H) = 5 + 23 = 11 39 4118 52 3 44 min Heap H trees(H) = 5 marks(H) = 3 marked Heaps and Priority Queues Advanced Data Structures - Arora 42
• 43.  Insert.  Create a new singleton tree.  Add to root list; update min pointer (if necessary). FIBONACCI HEAPS: INSERT 723 30 17 35 26 46 24 39 4118 52 3 44 21 insert 21 min Heap H Heaps and Priority Queues Advanced Data Structures - Arora 43
• 44.  Insert.  Create a new singleton tree.  Add to root list; update min pointer (if necessary). FIBONACCI HEAPS: INSERT 39 41 723 18 52 3 30 17 35 26 46 24 44 21 min Heap H insert 21 Heaps and Priority Queues Advanced Data Structures - Arora 44
• 45.  Actual cost. O(1)  Potential of heap H, (H) = trees(H) + 2  marks(H)  Change in potential. +1  Amortized cost. O(1) FIBONACCI HEAPS: INSERT ANALYSIS 39 41 7 18 52 3 30 17 35 26 46 24 44 2123 min Heap H Heaps and Priority Queues Advanced Data Structures - Arora 45
• 46.  Linking operation. Make larger root be a child of smaller root. LINKING OPERATION 39 4118 52 3 4477 56 24 15 tree T1 tree T2 39 4118 52 3 44 77 56 24 15 tree T' smaller rootlarger root still heap-ordere Heaps and Priority Queues Advanced Data Structures - Arora 46
• 47.  Union. Combine two Fibonacci heaps.  Representation. Root lists are circular, doubly linked lists. FIBONACCI HEAPS: UNION 39 41 717 18 52 3 30 23 35 26 46 24 44 21 min min Heap H' Heap H'' Heaps and Priority Queues Advanced Data Structures - Arora 47
• 48.  Union. Combine two Fibonacci heaps.  Representation. Root lists are circular, doubly linked lists. FIBONACCI HEAPS: UNION 39 41 717 18 52 3 30 23 35 26 46 24 44 21 min Heap H Heaps and Priority Queues Advanced Data Structures - Arora 48
• 49.  Actual cost. O(1)  Change in potential. 0  Amortized cost. O(1) FIBONACCI HEAPS: UNION (H) = trees(H) + 2  marks(H) potential function 39 41 717 18 52 3 30 23 35 26 46 24 44 21 min Heap H Heaps and Priority Queues Advanced Data Structures - Arora 49
• 50.  Delete min.  Delete min; meld its children into root list; update min.  Consolidate trees so that no two roots have same rank. FIBONACCI HEAPS: DELETE MIN 39 4118 52 3 44 1723 30 7 35 26 46 24 min Heaps and Priority Queues Advanced Data Structures - Arora 50
• 51.  Delete min.  Delete min; meld its children into root list; update min.  Consolidate trees so that no two roots have same rank.  This can be done using something like counting sort. FIBONACCI HEAPS: DELETE MIN 39 411723 18 52 30 7 35 26 46 24 44 min Heaps and Priority Queues Advanced Data Structures - Arora 51
• 52. FIBONACCI HEAPS: DELETE MIN 39 411723 18 52 30 7 35 26 46 24 44 min current Heaps and Priority Queues Advanced Data Structures - Arora 52
• 53. FIBONACCI HEAPS: DELETE MIN 39 411723 18 52 30 7 35 26 46 24 44 0 1 2 3 currentmin rank Heaps and Priority Queues Advanced Data Structures - Arora 53
• 54. FIBONACCI HEAPS: DELETE MIN 39 411723 18 52 30 7 35 26 46 24 44 0 1 2 3 min current rank Heaps and Priority Queues Advanced Data Structures - Arora 54
• 55. FIBONACCI HEAPS: DELETE MIN 39 411723 18 52 30 7 35 26 46 24 44 0 1 2 3 min current rank Heaps and Priority Queues Advanced Data Structures - Arora 55
• 56. FIBONACCI HEAPS: DELETE MIN 39 411723 18 52 30 7 35 26 46 24 44 0 1 2 3 min current rank link 23 into 17 Heaps and Priority Queues Advanced Data Structures - Arora 56
• 57. FIBONACCI HEAPS: DELETE MIN 39 4117 23 18 52 30 7 35 26 46 24 44 0 1 2 3 min current rank link 17 into 7 Heaps and Priority Queues Advanced Data Structures - Arora 57
• 58. FIBONACCI HEAPS: DELETE MIN 39 417 30 18 52 17 35 26 46 24 44 0 1 2 3 23 current min rank link 24 into 7 Heaps and Priority Queues Advanced Data Structures - Arora 58
• 59. FIBONACCI HEAPS: DELETE MIN 39 417 30 18 52 23 17 35 26 46 24 44 0 1 2 3 min current rank Heaps and Priority Queues Advanced Data Structures - Arora 59
• 60. FIBONACCI HEAPS: DELETE MIN 39 417 30 18 52 23 17 35 26 46 24 44 0 1 2 3 min current rank Heaps and Priority Queues Advanced Data Structures - Arora 60
• 61. FIBONACCI HEAPS: DELETE MIN 39 417 30 18 52 23 17 35 26 46 24 44 0 1 2 3 min current rank Heaps and Priority Queues Advanced Data Structures - Arora 61
• 62. FIBONACCI HEAPS: DELETE MIN 7 30 52 23 17 35 26 46 24 min 3941 18 44 stop Heaps and Priority Queues Advanced Data Structures - Arora 62
• 63.  Actual cost. O(rank(H)) + O(trees(H))  O(rank(H)) to meld min's children into root list.  O(rank(H)) + O(trees(H)) to update min.  O(rank(H)) + O(trees(H)) to consolidate trees.  Change in potential. O(rank(H)) - trees(H)  trees(H' )  rank(H) + 1 since no two trees have same rank.  Child nodes from the min root in H are moved to the list of roots in H’ and are no longer marked.  That is, marks(H’)  marks(H)  That is, (H)  trees(H’) – trees(H)  That is, (H)  rank(H) + 1 – trees(H).  Amortized cost. O(rank(H)). [We can scale the constants so that the constant of O notation matches 1.] FIBONACCI HEAPS: DELETE MIN ANALYSIS Heaps and Priority Queues Advanced Data Structures - Arora 63
• 64.  Is amortized cost of O(rank(H)) good?  Yes, if rank(H)  O(log n)  Therefore, our goal in Fibonacci trees is to bound rank(H). If only insert and delete-min operations, all trees are binomial trees and rank(H) is automatically bounded by lg(n).  We need to implement decrease key operation so that rank(H) = O(log n), as well as cost of decrease key is O(1). FIBONACCI HEAPS: DELETE MIN ANALYSIS Heaps and Priority Queues Advanced Data Structures - Arora 64
• 65.  Intuition for deceasing the key of node x.  If heap-order is not violated, just decrease the key of x.  Otherwise, cut tree rooted at x and meld into root list.  To keep trees flat: as soon as a node has its second child cut, cut it off and meld into root list (and unmark it). FIBONACCI HEAPS: DECREASE KEY 24 46 17 30 23 7 88 26 21 52 39 18 41 38 7235 min marked node: one child already cut Heaps and Priority Queues Advanced Data Structures - Arora 65
• 66.  If the parent node is already marked and the child node is cut, then we need to cut the parent node also, mark it’s parent. This is done recursively.  Suppose this continues for “c” levels. Then, the actual cost of decrease key operation is O(c). FIBONACCI HEAPS: DECREASE KEY (CONT.) Heaps and Priority Queues Advanced Data Structures - Arora 66
• 67.  Actual cost. O(c)  O(1) time for changing the key.  O(1) time for each of c cuts, plus melding into root list.  Change in potential. O(1) - c  trees(H') = trees(H) + c.  marks(H')  marks(H) - c + 2.    c + 2  (-c + 2) = 4 - c.  Amortized cost. O(1) This explains/motivates the potential function which puts twice as many units on marks as on trees. FIBONACCI HEAPS: DECREASE KEY ANALYSIS Heaps and Priority Queues Advanced Data Structures - Arora 67
• 68.  Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote its children in the order in which they were linked to x. Then:  Proof:  When yi was linked into x, x had at least i -1 children y1, …, yi-1.  Since only trees of equal rank are linked, at that time rank(yi) = rank(xi)  i - 1.  Since then, at most one child cut from yi, or it would have been cut  Thus, right now rank(yi)  i - 2. FIBONACCI HEAPS: BOUNDING THE RANK  rank (yi )  0 if i 1 i 2 if i 1    x y1 y2 yk… Heaps and Priority Queues Advanced Data Structures - Arora 68
• 69. • We want to bound how many nodes can be cut from a tree before it loses a “rank”. • Definition: Let Fk be smallest possible tree of rank k satisfying property. FIBONACCI HEAPS: BOUNDING THE RANK (CONT.) F0 F1 F2 F3 F4 F5 1 2 3 5 8 13 Heaps and Priority Queues Advanced Data Structures - Arora 69
• 70. • We want to bound how many nodes can be cut from a tree before it loses a “rank”. • Definition: Let Fk be smallest possible tree of rank k satisfying property. FIBONACCI HEAPS: BOUNDING THE RANK (CONT.) F4 F5 8 13 F6 8 + 13 = 21 Heaps and Priority Queues Advanced Data Structures - Arora 70
• 71.  Definition. Let Fk be smallest possible tree of rank k satisfying property.  Fibonacci fact. Fk  k, where  = (1 + 5) / 2  1.618.  Corollary. rank(H)  log n . FIBONACCI HEAPS: BOUNDING THE RANK Heaps and Priority Queues Advanced Data Structures - Arora 71
• 72.  Delete node x.  decrease-key of x to -.  delete-min element in heap.  Amortized cost. O(rank(H))  O(1) amortized for decrease-key.  O(rank(H)) amortized for delete-min. FIBONACCI HEAPS: RANDOM DELETE Heaps and Priority Queues Advanced Data Structures - Arora 72
• 73. PRIORITY QUEUES PERFORMANCE COST SUMMARY † amortizedn = number of elements in priority queue make-heap Operation insert find-min delete-min union decrease-key delete 1 Binary Heap log n 1 log n n log n log n 1 Binomial Heap log n log n log n log n log n log n 1 Fibonacci Heap † 1 1 log n 1 1 log n 1 Linked List 1 n n 1 n n is-empty 1 1 11 Heaps and Priority Queues Advanced Data Structures - Arora 73
• 74. Advanced Data Structures Basics Record / Struct Arrays / Linked Lists / Stacks / Queues Graphs / Trees / BSTs Advanced Trie, B-Tree Splay Trees R-Trees Heaps and PQs Union Find, Sets WHERE WE ARE Heaps and Priority Queues Advanced Data Structures - Arora 74