Successfully reported this slideshow.

Linked lists

757 views

Published on

Published in: Education, Technology, Business
  • Be the first to comment

Linked lists

  1. 1. Linked Lists: Locking, LockFree, and Beyond … Dr. C.V. Suresh Babu
  2. 2. Today: Concurrent Objects • Adding threads should not lower throughput – Contention effects – Mostly fixed by Queue locks • Should increase throughput – Not possible if inherently sequential – Surprising things are parallelizable Linked Lists 2
  3. 3. Coarse-Grained Synchronization • Each method locks the object – Avoid contention using queue locks Linked Lists 3
  4. 4. Coarse-Grained Synchronization • Each method locks the object – Avoid contention using queue locks – Easy to reason about • In simple cases Linked Lists 4
  5. 5. Coarse-Grained Synchronization • Each method locks the object – Avoid contention using queue locks – Easy to reason about • In simple cases • So, are we done? Linked Lists 5
  6. 6. Coarse-Grained Synchronization • Sequential bottleneck – Threads “stand in line” Linked Lists 6
  7. 7. Coarse-Grained Synchronization • Sequential bottleneck – Threads “stand in line” • Adding more threads – Does not improve throughput – Struggle to keep it from getting worse Linked Lists 7
  8. 8. Coarse-Grained Synchronization • Sequential bottleneck – Threads “stand in line” • Adding more threads – Does not improve throughput – Struggle to keep it from getting worse • So why even use a multiprocessor? – Well, some apps inherently parallel … Linked Lists 8
  9. 9. This Lecture • Introduce four “patterns” – Bag of tricks … – Methods that work more than once … Linked Lists 9
  10. 10. This Lecture • Introduce four “patterns” – Bag of tricks … – Methods that work more than once … • For highly-concurrent objects – Concurrent access – More threads, more throughput Linked Lists 10
  11. 11. First: Fine-Grained Synchronization • Instead of using a single lock … • Split object into – Independently-synchronized components • Methods conflict when they access – The same component … – At the same time Linked Lists 11
  12. 12. Second: Optimistic Synchronization • Search without locking … • If you find it, lock and check … – OK: we are done – Oops: start over • Evaluation – Usually cheaper than locking, but – Mistakes are expensive Linked Lists 12
  13. 13. Third: Lazy Synchronization • Postpone hard work • Removing components is tricky – Logical removal • Mark component to be deleted – Physical removal • Do what needs to be done Linked Lists 13
  14. 14. Fourth: Lock-Free Synchronization • Don’t use locks at all – Use compareAndSet() & relatives … • Advantages – No Scheduler Assumptions/Support • Disadvantages – Complex – Sometimes high overhead Linked Lists 14
  15. 15. Linked List • Illustrate these patterns … • Using a list-based Set – Common application – Building block for other apps Linked Lists 15
  16. 16. Set Interface • Unordered collection of items • No duplicates • Methods – add(x) put x in set – remove(x) take x out of set – contains(x) tests if x in set Linked Lists 16
  17. 17. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Linked Lists 17
  18. 18. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Add item to set Linked Lists 18
  19. 19. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(Tt x); } Remove item from set Linked Lists 19
  20. 20. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Is item in set? Linked Lists 20
  21. 21. List Node public class Node { public T item; public int key; public Node next; } Linked Lists 21
  22. 22. List Node public class Node { public T item; public int key; public Node next; } item of interest Linked Lists 22
  23. 23. List Node public class Node { public T item; public int key; public Node next; } Usually hash code Linked Lists 23
  24. 24. List Node public class Node { public T item; public int key; public Node next; } Reference to next node Linked Lists 24
  25. 25. The List-Based Set -∞ a b c +∞ Sorted with Sentinel nodes (min & max possible keys) Linked Lists 25
  26. 26. Invariants • Sentinel nodes – tail reachable from head • Sorted • No duplicates Linked Lists 26
  27. 27. Sequential List Based Set Add() a c d a b c Remove() Linked Lists 27
  28. 28. Sequential List Based Set Add() a c d b c b Remove() a Linked Lists 28
  29. 29. Coarse Grained Locking a b Linked Lists d 29
  30. 30. Coarse Grained Locking a d b c Linked Lists 30
  31. 31. Coarse Grained Locking a d b honk! honk! c Simple but hotspot + bottleneck Linked Lists 31
  32. 32. Coarse-Grained Locking • Easy, same as synchronized methods – “One lock to rule them all …” Linked Lists 32
  33. 33. Coarse-Grained Locking • Easy, same as synchronized methods – “One lock to rule them all …” • Simple, clearly correct – Deserves respect! • Works poorly with contention – Queue locks help – But bottleneck still an issue Linked Lists 33
  34. 34. Fine-grained Locking • Requires careful thought • Split object into pieces – Each piece has own lock – Methods that work on disjoint pieces need not exclude each other Linked Lists 34
  35. 35. Hand-over-Hand locking a b Linked Lists c 35
  36. 36. Hand-over-Hand locking a b Linked Lists c 36
  37. 37. Hand-over-Hand locking a b Linked Lists c 37
  38. 38. Hand-over-Hand locking a b Linked Lists c 38
  39. 39. Hand-over-Hand locking a b Linked Lists c 39
  40. 40. Removing a Node a b c d remove(b) Linked Lists 40
  41. 41. Removing a Node a b c d remove(b) Linked Lists 41
  42. 42. Removing a Node a b c d remove(b) Linked Lists 42
  43. 43. Removing a Node a b c d remove(b) Linked Lists 43
  44. 44. Removing a Node a b c d remove(b) Linked Lists 44
  45. 45. Removing a Node a remove(b) c d Why hold 2 locks? Linked Lists 45
  46. 46. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 46
  47. 47. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 47
  48. 48. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 48
  49. 49. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 49
  50. 50. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 50
  51. 51. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 51
  52. 52. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 52
  53. 53. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 53
  54. 54. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 54
  55. 55. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 55
  56. 56. Uh, Oh a c d remove(c) remove(b) Linked Lists 56
  57. 57. Uh, Oh Bad news, c not removed a c d remove(c) remove(b) Linked Lists 57
  58. 58. Problem • To delete node c – Swing node b’s next field to d a b c • Problem is, – Someone deleting b concurrently could direct a pointer a b c to c Linked Lists 58
  59. 59. Insight • If a node is locked – No one can delete node’s successor • If a thread locks – Node to be deleted – And its predecessor – Then it works Linked Lists 59
  60. 60. Hand-Over-Hand Again a b c d remove(b) Linked Lists 60
  61. 61. Hand-Over-Hand Again a b c d remove(b) Linked Lists 61
  62. 62. Hand-Over-Hand Again a b c d remove(b) Linked Lists 62
  63. 63. Hand-Over-Hand Again a b c d Found it! remove(b) Linked Lists 63
  64. 64. Hand-Over-Hand Again a b c d Found it! remove(b) Linked Lists 64
  65. 65. Hand-Over-Hand Again a c d remove(b) Linked Lists 65
  66. 66. Removing a Node a b c d remove(c) remove(b) Linked Lists 66
  67. 67. Removing a Node a b c d remove(c) remove(b) Linked Lists 67
  68. 68. Removing a Node a b c d remove(c) remove(b) Linked Lists 68
  69. 69. Removing a Node a b c d remove(c) remove(b) Linked Lists 69
  70. 70. Removing a Node a b c d remove(c) remove(b) Linked Lists 70
  71. 71. Removing a Node a b c d remove(c) remove(b) Linked Lists 71
  72. 72. Removing a Node a b c d remove(c) remove(b) Linked Lists 72
  73. 73. Removing a Node a b c d remove(c) remove(b) Linked Lists 73
  74. 74. Removing a Node a b c d remove(c) Must acquire Lock of b Linked Lists 74
  75. 75. Removing a Node a b c d remove(c) Cannot acquire lock of b Linked Lists 75
  76. 76. Removing a Node a b c d remove(c) Wait! Linked Lists 76
  77. 77. Removing a Node a b d Proceed to remove(b) Linked Lists 77
  78. 78. Removing a Node a b d remove(b) Linked Lists 78
  79. 79. Removing a Node a b d remove(b) Linked Lists 79
  80. 80. Removing a Node a d remove(b) Linked Lists 80
  81. 81. Removing a Node a d Linked Lists 81
  82. 82. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Linked Lists 82
  83. 83. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Key used to order node Linked Lists 83
  84. 84. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { currNode.unlock(); predNode.unlock(); }} Predecessor and current nodes Linked Lists 84
  85. 85. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { Make sure … locks released } finally { curr.unlock(); pred.unlock(); }} Linked Lists 85
  86. 86. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); Everything else pred.unlock(); }} Linked Lists 86
  87. 87. Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists 87
  88. 88. Remove method lock pred == head try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists 88
  89. 89. Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists Lock current 89
  90. 90. Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists Traversing list 90
  91. 91. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 91
  92. 92. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } Search key range pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 92
  93. 93. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); At start of each loop: pred = curr; curr and pred locked curr = curr.next; curr.lock(); } return false; Linked Lists 93
  94. 94. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return found, If item false; remove node Linked Lists 94
  95. 95. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; remove it If node found, Linked Lists 95
  96. 96. Remove: searching Unlock predecessor while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 96
  97. 97. Remove: searching Only one node locked! while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 97
  98. 98. Remove: searching while (curr.key <= key) { demote current if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 98
  99. 99. Remove: searching while (curr.key <= key) { Find and lock new current if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Linked Lists 99
  100. 100. Remove: searching while (curr.key <= key) { Lock invariant restored { if (item == curr.item) pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Linked Lists 100
  101. 101. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; Otherwise, not present } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 101
  102. 102. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; •pred reachable from head curr.lock(); •curr is pred.next } •So curr.item is in the set return false; Linked Lists 103
  103. 103. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Linearization point if return false; item is present Linked Lists 104
  104. 104. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Node locked, so no other return false; thread can remove it …. Linked Lists 105
  105. 105. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; Item not present curr.lock(); } return false; Linked Lists 106
  106. 106. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next;•pred reachable from head •curr is pred.next curr.lock(); •pred.key < key } •key < curr.key return false; Linked Lists 107
  107. 107. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } Linearization point pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 108
  108. 108. Adding Nodes • To add node e – Must lock predecessor – Must lock successor • Neither can be deleted – (Is successor lock actually required?) Linked Lists 109
  109. 109. Drawbacks • Better than coarse-grained lock – Threads can traverse in parallel • Still not ideal – Long chain of acquire/release – Inefficient Linked Lists 110
  110. 110. Optimistic Synchronization • Find nodes without locking • Lock nodes • Check that everything is OK Linked Lists 111
  111. 111. Optimistic: Traverse without Locking a b d e Aha! add(c) Linked Lists 112
  112. 112. Optimistic: Lock and Load a b d e add(c) Linked Lists 113
  113. 113. Optimistic: Lock and Load c a b d e add(c) Linked Lists 114
  114. 114. What could go wrong? a b d e Aha! add(c) Linked Lists 115
  115. 115. What could go wrong? a b d e add(c) Linked Lists 116
  116. 116. What could go wrong? a b d e remove(b ) Linked Lists 117
  117. 117. What could go wrong? a b d e remove(b ) Linked Lists 118
  118. 118. What could go wrong? a b d e add(c) Linked Lists 119
  119. 119. What could go wrong? c a b d e add(c) Linked Lists 120
  120. 120. What could go wrong? a add(c) d e Uh-oh Linked Lists 121
  121. 121. Validate – Part 1 a b d e Yes, b still reachable from head add(c) Linked Lists 122
  122. 122. What Else Could Go Wrong? a b d e Aha! add(c) Linked Lists 123
  123. 123. What Else Coould Go Wrong? a b d e add(b’) add(c) Linked Lists 124
  124. 124. What Else Coould Go Wrong? a b d b’ e add(b’) add(c) Linked Lists 125
  125. 125. What Else Could Go Wrong? a b d e b’ add(c) Linked Lists 126
  126. 126. What Else Could Go Wrong? c a b d e add(c) Linked Lists 127
  127. 127. Validate Part 2 (while holding locks) a b d e Yes, b still points to d add(c) Linked Lists 128
  128. 128. Optimistic: Linearization Point a b d e c add(c) Linked Lists 129
  129. 129. Correctness • If – Nodes b and c both locked – Node b still accessible – Node c still successor to b • Then – Neither will be deleted – OK to delete and return true Linked Lists 130
  130. 130. Unsuccessful Remove a b d e Aha! remove(c) Linked Lists 131
  131. 131. Validate (1) a b d e Yes, b still reachable from head remove(c) Linked Lists 132
  132. 132. Validate (2) a b d e Yes, b still points to d remove(c) Linked Lists 133
  133. 133. OK Computer a b remove(c) d e return false Linked Lists 134
  134. 134. Correctness • If – Nodes b and d both locked – Node b still accessible – Node d still successor to b • Then – Neither will be deleted – No thread can add c after b – OK to return false Linked Lists 135
  135. 135. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Linked Lists 136
  136. 136. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Predecessor & return false; current nodes } Linked Lists 137
  137. 137. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Begin at the return false; } beginning Linked Lists 138
  138. 138. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Search range of keys return false; } Linked Lists 139
  139. 139. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; Predecessor reachable } Linked Lists 140
  140. 140. Validation private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; Is current node next? } Linked Lists 141
  141. 141. Validation private boolean Otherwise move on validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Linked Lists 142
  142. 142. Validation private boolean Predecessor not reachable validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Linked Lists 143
  143. 143. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Linked Lists 144
  144. 144. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Search key Linked Lists 145
  145. 145. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Retry on synchronization conflict Linked Lists 146
  146. 146. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Examine predecessor and current nodes } … Linked Lists 147
  147. 147. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Search by key } … Linked Lists 148
  148. 148. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Stop if we find item } … Linked Lists 149
  149. 149. Remove: searching public boolean remove(Item item) { Move along int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Linked Lists 150
  150. 150. On Exit from Loop • If item is present – curr holds item – pred just before curr • If item is absent – curr has first higher key – pred just before curr • Assuming no synchronization problems Linked Lists 151
  151. 151. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 152
  152. 152. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; Always unlock }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 153
  153. 153. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; Lock both nodes }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 154
  154. 154. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; Check for synchronization } else { return false; conflicts }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 155
  155. 155. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; target found, }}} finally { remove node pred.unlock(); curr.unlock(); }}} Linked Lists 156
  156. 156. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; target not found } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 157
  157. 157. Optimistic List • Limited hot-spots – Targets of add(), remove(), contains() – No contention on traversals • Moreover – Traversals are wait-free – Food for thought … Linked Lists 158
  158. 158. So Far, So Good • Much less lock acquisition/release – Performance – Concurrency • Problems – Need to traverse list twice – contains() method acquires locks Linked Lists 159
  159. 159. Evaluation • Optimistic is effective if – cost of scanning twice without locks is less than – cost of scanning once with locks • Drawback – contains() acquires locks – 90% of calls in many apps Linked Lists 160
  160. 160. Lazy List • Like optimistic, except – Scan once – contains(x) never locks … • Key insight – Removing nodes causes trouble – Do it “lazily” Linked Lists 161
  161. 161. Lazy List • remove() – Scans list (as before) – Locks predecessor & current (as before) • Logical delete – Marks current node as removed (new!) • Physical delete – Redirects predecessor’s next (as before) Linked Lists 162
  162. 162. Lazy Removal a b Linked Lists c d 163
  163. 163. Lazy Removal a b c d Present in list Linked Lists 164
  164. 164. Lazy Removal a b c d Logically deleted Linked Lists 165
  165. 165. Lazy Removal a b c d Physically deleted Linked Lists 166
  166. 166. Lazy Removal a b d Physically deleted Linked Lists 167
  167. 167. Lazy List • All Methods – Scan through locked and marked nodes – Removing a node doesn’t slow down other method calls … • Must still lock pred and curr nodes. Linked Lists 168
  168. 168. Validation • • • • No need to rescan list! Check that pred is not marked Check that curr is not marked Check that pred points to curr Linked Lists 169
  169. 169. Business as Usual a b Linked Lists c 170
  170. 170. Business as Usual a b Linked Lists c 171
  171. 171. Business as Usual a b Linked Lists c 172
  172. 172. Business as Usual a b c remove(b) Linked Lists 173
  173. 173. Business as Usual a b c a not marked Linked Lists 174
  174. 174. Business as Usual a b c a still points to b Linked Lists 175
  175. 175. Business as Usual a b c Logical delete Linked Lists 176
  176. 176. Business as Usual a b c physical delete Linked Lists 177
  177. 177. Business as Usual a b Linked Lists c 178
  178. 178. Validation private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Linked Lists 179
  179. 179. List Validate Method private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Predecessor not Logically removed Linked Lists 180
  180. 180. List Validate Method private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Current not Logically removed Linked Lists 181
  181. 181. List Validate Method private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Predecessor still Points to current Linked Lists 182
  182. 182. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 183
  183. 183. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; Validate as before } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 184
  184. 184. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; Key found }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 185
  185. 185. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); Logical remove curr.unlock(); }}} Linked Lists 186
  186. 186. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); physical remove curr.unlock(); }}} Linked Lists 187
  187. 187. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Linked Lists 188
  188. 188. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Start at the head Linked Lists 189
  189. 189. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Search key range Linked Lists 190
  190. 190. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Traverse without locking (nodes may have been removed) Linked Lists 191
  191. 191. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Present and undeleted? Linked Lists 192
  192. 192. Summary: Wait-free Contains a 0 b 0 d 1 c0 e 0 Use Mark bit + list ordering 1. Not marked  in the set 2. Marked or missing  not in the set Linked Lists 193
  193. 193. Lazy List a 0 b 0 d 1 c0 e 0 Lazy add() and remove() + Wait-free contains() Linked Lists 194
  194. 194. Evaluation • Good: – – – – contains() doesn’t lock In fact, its wait-free! Good because typically high % contains() Uncontended calls don’t re-traverse • Bad – Contended add() and remove() calls do re-traverse – Traffic jam if one thread delays Linked Lists 195
  195. 195. Traffic Jam • Any concurrent data structure based on mutual exclusion has a weakness • If one thread – Enters critical section – And “eats the big muffin” • Cache miss, page fault, descheduled … – Everyone else using that lock is stuck! – Need to trust the scheduler…. Linked Lists 196
  196. 196. Reminder: Lock-Free Data Structures • No matter what … – Guarantees minimal progress in any execution – i.e. Some thread will always complete a method call – Even if others halt at malicious times – Implies that implementation can’t use locks Linked Lists 197
  197. 197. Lock-free Lists • Next logical step – Wait-free contains() – lock-free add() and remove() • Use only compareAndSet() – What could go wrong? Linked Lists 198
  198. 198. Remove Using CAS Logical Removal = Set Mark Bit a 0 b 0 Use CAS to verify pointer is correct Not enough! Linked Lists c 1 c0 e 0 Physical Removal CAS pointer 199
  199. 199. Problem… Logical Removal = Set Mark Bit a 0 b 0 c 1 c0 Problem: Physical d not added to list… Removal Must Prevent CAS manipulation of removed node’s pointer Linked Lists e 0 d 0 Node added Before Physical Removal CAS 200
  200. 200. The Solution: Combine Bit and Pointer Logical Removal = Set Mark Bit a 0 b 0 e 0 c 1 c0 d 0 Physical Removal Fail CAS: Node not added after logical CAS Removal Mark-Bit and Pointer are CASed together (AtomicMarkableReference) Linked Lists 201
  201. 201. Solution • Use AtomicMarkableReference • Atomically – Swing reference and – Update flag • Remove in two steps – Set mark bit in next field – Redirect predecessor’s pointer Linked Lists 202
  202. 202. Marking a Node • AtomicMarkableReference class – Java.util.concurrent.atomic package Reference address F mark bit Linked Lists 203
  203. 203. Extracting Reference & Mark Public Object get(boolean[] marked); Linked Lists 204
  204. 204. Extracting Reference & Mark Public Object get(boolean[] marked); Returns reference Returns mark at array index 0! Linked Lists 205
  205. 205. Extracting Mark Only public boolean isMarked(); Value of mark Linked Lists 206
  206. 206. Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); Linked Lists 207
  207. 207. Changing State If this is the current reference … Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); And this is the current mark … Linked Lists 208
  208. 208. Changing State …then change to this new reference … Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); … and this new mark Linked Lists 209
  209. 209. Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); Linked Lists 210
  210. 210. Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); If this is the current reference … Linked Lists 211
  211. 211. Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); .. then change to this new mark. Linked Lists 212
  212. 212. Removing a Node a CAS b c d remov e c Linked Lists 213
  213. 213. Removing a Node failed CAS a CAS b c d remov e c remov e b Linked Lists 214
  214. 214. Removing a Node a b c d remov e c remov e b Linked Lists 215
  215. 215. Removing a Node a d remov e c remov e b Linked Lists 216
  216. 216. Traversing the List • Q: what do you do when you find a “logically” deleted node in your path? • A: finish the job. – CAS the predecessor’s next field – Proceed (repeat as needed) Linked Lists 217
  217. 217. Lock-Free Traversal (only Add and Remove) pred curr pred a CAS curr b c d Uh-oh Linked Lists 218
  218. 218. The Window Class class Window public Node public Node Window(Node this.pred } } { pred; curr; pred, Node curr) { = pred; this.curr = curr; Linked Lists 219
  219. 219. The Window Class class Window public Node public Node Window(Node this.pred } } { pred; curr; pred, Node curr) { = pred; this.curr = curr; A container for pred and current values Linked Lists 220
  220. 220. Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Linked Lists 221
  221. 221. Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Find returns window Linked Lists 222
  222. 222. Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Extract pred and curr Linked Lists 223
  223. 223. The Find Method Window window = find(item); item At some instant, pred curr or … succ Linked Lists© Herlihy-Shavit 2007 224
  224. 224. The Find Method Window window = find(item); At some instant, pred curr= null item not in list succ Linked Lists© Herlihy-Shavit 2007 225
  225. 225. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 226
  226. 226. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists Keep trying 227
  227. 227. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Find neighbors Linked Lists 228
  228. 228. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} She’s not there … Linked Lists 229
  229. 229. Remove public boolean remove(T item) { Boolean Try to mark node as deleted snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 230
  230. 230. Remove public boolean remove(T item) { Boolean snip; If it doesn’t while just { work,(true) retry, Window window = find(head, key); if it does, job Node pred = window.pred, curr = window.curr; if (curr.key != essentially donekey) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 231
  231. 231. Remove public boolean remove(T item) { Boolean snip; while (true) { a Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; Try to advance reference } else { (if succ = curr.next.getReference(); Node we don’t succeed, someone else did or will). snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 232
  232. 232. Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Linked Lists 233
  233. 233. Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Item already there. Linked Lists 234
  234. 234. Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} create new node Linked Lists 235
  235. 235. Add public boolean add(T item) { boolean splice; Install new node, while (true) { else retry loop Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Linked Lists 236
  236. 236. Wait-free Contains public boolean contains(Tt item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) } Linked Lists 237
  237. 237. Wait-free Contains public boolean contains(T item) { Only diff is that we boolean marked; get and check int key = item.hashCode(); Node curr = this.head; marked while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) } Linked Lists 238
  238. 238. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 239
  239. 239. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); If list changes while (true) { while succ = curr.next.get(marked); while (marked[0]) { traversed, … start over } Lock-Free if (curr.key >= key) because we return new Window(pred, curr); pred = curr; start over only curr = succ; if someone else } }} Linked Lists makes progress 240
  240. 240. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr looking succ = head Start = null, from null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 241
  241. 241. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { Move down the list pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 242
  242. 242. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = Get ref to successor succ; } }} and current deleted bit Linked Lists 243
  243. 243. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } Try }} if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; to remove deleted nodes in } path…code details soon Linked Lists 244
  244. 244. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); Ifwhile (true)that is greater or curr key { succ curr.next.get(marked); equal,= return pred and curr while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 245
  245. 245. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); Otherwise advance window and while (marked[0]) { … loop again } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 246
  246. 246. Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 247
  247. 247. Lock-free Find Try to snip out node retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 248
  248. 248. Lock-free Find if predecessor’s next field changed must retry whole traversal { retry: while (true) … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 249
  249. 249. Lock-free Find Otherwise move on to check if next node deleted retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 250
  250. 250. Performance On 16 node shared memory machine Benchmark throughput of Java List-based Set algs. Vary % of Contains() method Calls. Linked Lists 251
  251. 251. High Contains Ratio Lock-free Lazy list Course Grained Fine Lock-coupling Linked Lists 252
  252. 252. Low Contains Ratio Lock-free Lazy list Course Grained Fine Lock-coupling Linked Lists 253
  253. 253. As Contains Ratio Increases Lock-free Lazy list Course Grained Fine Lock-coupling % Contains() Linked Lists 254
  254. 254. Summary • • • • Coarse-grained locking Fine-grained locking Optimistic synchronization Lock-free synchronization Linked Lists 255

×