Linked lists

  • 185 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
185
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • We can improve concurrency by locking individual entries, rather than locking the list as a whole. Instead of placing a lock on the entire list, let us add a lock to each entry, along with lock() and unlock() methods. As a thread traverses the list, it locks each entry when it first visits, and sometime later releases it. Suchfine-grained locking permits concurrent threads to traverse the list together in a pipelined fashion.
  • Here is a naive approach to removing a node. Each thread locks the node being examined and its predecessor.
  • Here is a naive approach to removing a node. Each thread locks the node being examined and its predecessor.
  • Supposed we only locked one node and not two. Consider two concurrent threads that want to remove adjacent nodes.
  • The red thread looks for the predecessor to c, while the green thread looks for the predecessor to b.
  • But the final effect is to remove b but not c!
  • But the final effect is to remove b but not c!
  • Notice that we could also have the linearization points be those when the locks are acquired.
  • successor lock not actually required!
  • If we don’t succeed then either someone already removed curr, or someone modified pred, for example, by inserting another node as pred’s successor, in which case we need not worry since some other later thread that will attempt to traverse the list will remove the marked curr node before traversing beyond it.
  • The wait-free mContains{} method of the cLockFreeList{} is
    the same as that of the cLazyList{} with one small change: to test if
    fCurr{} is marked we apply lstinline+curr.next.get(marked)+
    and check that aMarked{0} is true.
  • The wait-free mContains{} method of the cLockFreeList{} is
    the same as that of the cLazyList{} with one small change: to test if
    fCurr{} is marked we apply lstinline+curr.next.get(marked)+
    and check that aMarked{0} is true.
  • We saw 4 approaches to concurrent data structure design.

Transcript

  • 1. Linked Lists: Locking, LockFree, and Beyond … Dr. C.V. Suresh Babu
  • 2. Today: Concurrent Objects • Adding threads should not lower throughput – Contention effects – Mostly fixed by Queue locks • Should increase throughput – Not possible if inherently sequential – Surprising things are parallelizable Linked Lists 2
  • 3. Coarse-Grained Synchronization • Each method locks the object – Avoid contention using queue locks Linked Lists 3
  • 4. Coarse-Grained Synchronization • Each method locks the object – Avoid contention using queue locks – Easy to reason about • In simple cases Linked Lists 4
  • 5. Coarse-Grained Synchronization • Each method locks the object – Avoid contention using queue locks – Easy to reason about • In simple cases • So, are we done? Linked Lists 5
  • 6. Coarse-Grained Synchronization • Sequential bottleneck – Threads “stand in line” Linked Lists 6
  • 7. Coarse-Grained Synchronization • Sequential bottleneck – Threads “stand in line” • Adding more threads – Does not improve throughput – Struggle to keep it from getting worse Linked Lists 7
  • 8. Coarse-Grained Synchronization • Sequential bottleneck – Threads “stand in line” • Adding more threads – Does not improve throughput – Struggle to keep it from getting worse • So why even use a multiprocessor? – Well, some apps inherently parallel … Linked Lists 8
  • 9. This Lecture • Introduce four “patterns” – Bag of tricks … – Methods that work more than once … Linked Lists 9
  • 10. This Lecture • Introduce four “patterns” – Bag of tricks … – Methods that work more than once … • For highly-concurrent objects – Concurrent access – More threads, more throughput Linked Lists 10
  • 11. First: Fine-Grained Synchronization • Instead of using a single lock … • Split object into – Independently-synchronized components • Methods conflict when they access – The same component … – At the same time Linked Lists 11
  • 12. Second: Optimistic Synchronization • Search without locking … • If you find it, lock and check … – OK: we are done – Oops: start over • Evaluation – Usually cheaper than locking, but – Mistakes are expensive Linked Lists 12
  • 13. Third: Lazy Synchronization • Postpone hard work • Removing components is tricky – Logical removal • Mark component to be deleted – Physical removal • Do what needs to be done Linked Lists 13
  • 14. Fourth: Lock-Free Synchronization • Don’t use locks at all – Use compareAndSet() & relatives … • Advantages – No Scheduler Assumptions/Support • Disadvantages – Complex – Sometimes high overhead Linked Lists 14
  • 15. Linked List • Illustrate these patterns … • Using a list-based Set – Common application – Building block for other apps Linked Lists 15
  • 16. Set Interface • Unordered collection of items • No duplicates • Methods – add(x) put x in set – remove(x) take x out of set – contains(x) tests if x in set Linked Lists 16
  • 17. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Linked Lists 17
  • 18. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Add item to set Linked Lists 18
  • 19. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(Tt x); } Remove item from set Linked Lists 19
  • 20. List-Based Sets public interface Set<T> { public boolean add(T x); public boolean remove(T x); public boolean contains(T x); } Is item in set? Linked Lists 20
  • 21. List Node public class Node { public T item; public int key; public Node next; } Linked Lists 21
  • 22. List Node public class Node { public T item; public int key; public Node next; } item of interest Linked Lists 22
  • 23. List Node public class Node { public T item; public int key; public Node next; } Usually hash code Linked Lists 23
  • 24. List Node public class Node { public T item; public int key; public Node next; } Reference to next node Linked Lists 24
  • 25. The List-Based Set -∞ a b c +∞ Sorted with Sentinel nodes (min & max possible keys) Linked Lists 25
  • 26. Invariants • Sentinel nodes – tail reachable from head • Sorted • No duplicates Linked Lists 26
  • 27. Sequential List Based Set Add() a c d a b c Remove() Linked Lists 27
  • 28. Sequential List Based Set Add() a c d b c b Remove() a Linked Lists 28
  • 29. Coarse Grained Locking a b Linked Lists d 29
  • 30. Coarse Grained Locking a d b c Linked Lists 30
  • 31. Coarse Grained Locking a d b honk! honk! c Simple but hotspot + bottleneck Linked Lists 31
  • 32. Coarse-Grained Locking • Easy, same as synchronized methods – “One lock to rule them all …” Linked Lists 32
  • 33. Coarse-Grained Locking • Easy, same as synchronized methods – “One lock to rule them all …” • Simple, clearly correct – Deserves respect! • Works poorly with contention – Queue locks help – But bottleneck still an issue Linked Lists 33
  • 34. Fine-grained Locking • Requires careful thought • Split object into pieces – Each piece has own lock – Methods that work on disjoint pieces need not exclude each other Linked Lists 34
  • 35. Hand-over-Hand locking a b Linked Lists c 35
  • 36. Hand-over-Hand locking a b Linked Lists c 36
  • 37. Hand-over-Hand locking a b Linked Lists c 37
  • 38. Hand-over-Hand locking a b Linked Lists c 38
  • 39. Hand-over-Hand locking a b Linked Lists c 39
  • 40. Removing a Node a b c d remove(b) Linked Lists 40
  • 41. Removing a Node a b c d remove(b) Linked Lists 41
  • 42. Removing a Node a b c d remove(b) Linked Lists 42
  • 43. Removing a Node a b c d remove(b) Linked Lists 43
  • 44. Removing a Node a b c d remove(b) Linked Lists 44
  • 45. Removing a Node a remove(b) c d Why hold 2 locks? Linked Lists 45
  • 46. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 46
  • 47. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 47
  • 48. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 48
  • 49. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 49
  • 50. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 50
  • 51. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 51
  • 52. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 52
  • 53. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 53
  • 54. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 54
  • 55. Concurrent Removes a b c d remove(c) remove(b) Linked Lists 55
  • 56. Uh, Oh a c d remove(c) remove(b) Linked Lists 56
  • 57. Uh, Oh Bad news, c not removed a c d remove(c) remove(b) Linked Lists 57
  • 58. Problem • To delete node c – Swing node b’s next field to d a b c • Problem is, – Someone deleting b concurrently could direct a pointer a b c to c Linked Lists 58
  • 59. Insight • If a node is locked – No one can delete node’s successor • If a thread locks – Node to be deleted – And its predecessor – Then it works Linked Lists 59
  • 60. Hand-Over-Hand Again a b c d remove(b) Linked Lists 60
  • 61. Hand-Over-Hand Again a b c d remove(b) Linked Lists 61
  • 62. Hand-Over-Hand Again a b c d remove(b) Linked Lists 62
  • 63. Hand-Over-Hand Again a b c d Found it! remove(b) Linked Lists 63
  • 64. Hand-Over-Hand Again a b c d Found it! remove(b) Linked Lists 64
  • 65. Hand-Over-Hand Again a c d remove(b) Linked Lists 65
  • 66. Removing a Node a b c d remove(c) remove(b) Linked Lists 66
  • 67. Removing a Node a b c d remove(c) remove(b) Linked Lists 67
  • 68. Removing a Node a b c d remove(c) remove(b) Linked Lists 68
  • 69. Removing a Node a b c d remove(c) remove(b) Linked Lists 69
  • 70. Removing a Node a b c d remove(c) remove(b) Linked Lists 70
  • 71. Removing a Node a b c d remove(c) remove(b) Linked Lists 71
  • 72. Removing a Node a b c d remove(c) remove(b) Linked Lists 72
  • 73. Removing a Node a b c d remove(c) remove(b) Linked Lists 73
  • 74. Removing a Node a b c d remove(c) Must acquire Lock of b Linked Lists 74
  • 75. Removing a Node a b c d remove(c) Cannot acquire lock of b Linked Lists 75
  • 76. Removing a Node a b c d remove(c) Wait! Linked Lists 76
  • 77. Removing a Node a b d Proceed to remove(b) Linked Lists 77
  • 78. Removing a Node a b d remove(b) Linked Lists 78
  • 79. Removing a Node a b d remove(b) Linked Lists 79
  • 80. Removing a Node a d remove(b) Linked Lists 80
  • 81. Removing a Node a d Linked Lists 81
  • 82. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Linked Lists 82
  • 83. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Key used to order node Linked Lists 83
  • 84. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { currNode.unlock(); predNode.unlock(); }} Predecessor and current nodes Linked Lists 84
  • 85. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { Make sure … locks released } finally { curr.unlock(); pred.unlock(); }} Linked Lists 85
  • 86. Remove method public boolean remove(Item item) { int key = item.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); Everything else pred.unlock(); }} Linked Lists 86
  • 87. Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists 87
  • 88. Remove method lock pred == head try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists 88
  • 89. Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists Lock current 89
  • 90. Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Linked Lists Traversing list 90
  • 91. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 91
  • 92. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } Search key range pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 92
  • 93. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); At start of each loop: pred = curr; curr and pred locked curr = curr.next; curr.lock(); } return false; Linked Lists 93
  • 94. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return found, If item false; remove node Linked Lists 94
  • 95. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; remove it If node found, Linked Lists 95
  • 96. Remove: searching Unlock predecessor while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 96
  • 97. Remove: searching Only one node locked! while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 97
  • 98. Remove: searching while (curr.key <= key) { demote current if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 98
  • 99. Remove: searching while (curr.key <= key) { Find and lock new current if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Linked Lists 99
  • 100. Remove: searching while (curr.key <= key) { Lock invariant restored { if (item == curr.item) pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Linked Lists 100
  • 101. Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; Otherwise, not present } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 101
  • 102. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; •pred reachable from head curr.lock(); •curr is pred.next } •So curr.item is in the set return false; Linked Lists 103
  • 103. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Linearization point if return false; item is present Linked Lists 104
  • 104. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Node locked, so no other return false; thread can remove it …. Linked Lists 105
  • 105. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; Item not present curr.lock(); } return false; Linked Lists 106
  • 106. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next;•pred reachable from head •curr is pred.next curr.lock(); •pred.key < key } •key < curr.key return false; Linked Lists 107
  • 107. Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } Linearization point pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linked Lists 108
  • 108. Adding Nodes • To add node e – Must lock predecessor – Must lock successor • Neither can be deleted – (Is successor lock actually required?) Linked Lists 109
  • 109. Drawbacks • Better than coarse-grained lock – Threads can traverse in parallel • Still not ideal – Long chain of acquire/release – Inefficient Linked Lists 110
  • 110. Optimistic Synchronization • Find nodes without locking • Lock nodes • Check that everything is OK Linked Lists 111
  • 111. Optimistic: Traverse without Locking a b d e Aha! add(c) Linked Lists 112
  • 112. Optimistic: Lock and Load a b d e add(c) Linked Lists 113
  • 113. Optimistic: Lock and Load c a b d e add(c) Linked Lists 114
  • 114. What could go wrong? a b d e Aha! add(c) Linked Lists 115
  • 115. What could go wrong? a b d e add(c) Linked Lists 116
  • 116. What could go wrong? a b d e remove(b ) Linked Lists 117
  • 117. What could go wrong? a b d e remove(b ) Linked Lists 118
  • 118. What could go wrong? a b d e add(c) Linked Lists 119
  • 119. What could go wrong? c a b d e add(c) Linked Lists 120
  • 120. What could go wrong? a add(c) d e Uh-oh Linked Lists 121
  • 121. Validate – Part 1 a b d e Yes, b still reachable from head add(c) Linked Lists 122
  • 122. What Else Could Go Wrong? a b d e Aha! add(c) Linked Lists 123
  • 123. What Else Coould Go Wrong? a b d e add(b’) add(c) Linked Lists 124
  • 124. What Else Coould Go Wrong? a b d b’ e add(b’) add(c) Linked Lists 125
  • 125. What Else Could Go Wrong? a b d e b’ add(c) Linked Lists 126
  • 126. What Else Could Go Wrong? c a b d e add(c) Linked Lists 127
  • 127. Validate Part 2 (while holding locks) a b d e Yes, b still points to d add(c) Linked Lists 128
  • 128. Optimistic: Linearization Point a b d e c add(c) Linked Lists 129
  • 129. Correctness • If – Nodes b and c both locked – Node b still accessible – Node c still successor to b • Then – Neither will be deleted – OK to delete and return true Linked Lists 130
  • 130. Unsuccessful Remove a b d e Aha! remove(c) Linked Lists 131
  • 131. Validate (1) a b d e Yes, b still reachable from head remove(c) Linked Lists 132
  • 132. Validate (2) a b d e Yes, b still points to d remove(c) Linked Lists 133
  • 133. OK Computer a b remove(c) d e return false Linked Lists 134
  • 134. Correctness • If – Nodes b and d both locked – Node b still accessible – Node d still successor to b • Then – Neither will be deleted – No thread can add c after b – OK to return false Linked Lists 135
  • 135. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Linked Lists 136
  • 136. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Predecessor & return false; current nodes } Linked Lists 137
  • 137. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Begin at the return false; } beginning Linked Lists 138
  • 138. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Search range of keys return false; } Linked Lists 139
  • 139. Validation private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; Predecessor reachable } Linked Lists 140
  • 140. Validation private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; Is current node next? } Linked Lists 141
  • 141. Validation private boolean Otherwise move on validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Linked Lists 142
  • 142. Validation private boolean Predecessor not reachable validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Linked Lists 143
  • 143. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Linked Lists 144
  • 144. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Search key Linked Lists 145
  • 145. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Retry on synchronization conflict Linked Lists 146
  • 146. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Examine predecessor and current nodes } … Linked Lists 147
  • 147. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Search by key } … Linked Lists 148
  • 148. Remove: searching public boolean remove(Item item) { int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Stop if we find item } … Linked Lists 149
  • 149. Remove: searching public boolean remove(Item item) { Move along int key = item.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } … Linked Lists 150
  • 150. On Exit from Loop • If item is present – curr holds item – pred just before curr • If item is absent – curr has first higher key – pred just before curr • Assuming no synchronization problems Linked Lists 151
  • 151. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 152
  • 152. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; Always unlock }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 153
  • 153. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; Lock both nodes }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 154
  • 154. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; Check for synchronization } else { return false; conflicts }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 155
  • 155. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; target found, }}} finally { remove node pred.unlock(); curr.unlock(); }}} Linked Lists 156
  • 156. Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; target not found } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 157
  • 157. Optimistic List • Limited hot-spots – Targets of add(), remove(), contains() – No contention on traversals • Moreover – Traversals are wait-free – Food for thought … Linked Lists 158
  • 158. So Far, So Good • Much less lock acquisition/release – Performance – Concurrency • Problems – Need to traverse list twice – contains() method acquires locks Linked Lists 159
  • 159. Evaluation • Optimistic is effective if – cost of scanning twice without locks is less than – cost of scanning once with locks • Drawback – contains() acquires locks – 90% of calls in many apps Linked Lists 160
  • 160. Lazy List • Like optimistic, except – Scan once – contains(x) never locks … • Key insight – Removing nodes causes trouble – Do it “lazily” Linked Lists 161
  • 161. Lazy List • remove() – Scans list (as before) – Locks predecessor & current (as before) • Logical delete – Marks current node as removed (new!) • Physical delete – Redirects predecessor’s next (as before) Linked Lists 162
  • 162. Lazy Removal a b Linked Lists c d 163
  • 163. Lazy Removal a b c d Present in list Linked Lists 164
  • 164. Lazy Removal a b c d Logically deleted Linked Lists 165
  • 165. Lazy Removal a b c d Physically deleted Linked Lists 166
  • 166. Lazy Removal a b d Physically deleted Linked Lists 167
  • 167. Lazy List • All Methods – Scan through locked and marked nodes – Removing a node doesn’t slow down other method calls … • Must still lock pred and curr nodes. Linked Lists 168
  • 168. Validation • • • • No need to rescan list! Check that pred is not marked Check that curr is not marked Check that pred points to curr Linked Lists 169
  • 169. Business as Usual a b Linked Lists c 170
  • 170. Business as Usual a b Linked Lists c 171
  • 171. Business as Usual a b Linked Lists c 172
  • 172. Business as Usual a b c remove(b) Linked Lists 173
  • 173. Business as Usual a b c a not marked Linked Lists 174
  • 174. Business as Usual a b c a still points to b Linked Lists 175
  • 175. Business as Usual a b c Logical delete Linked Lists 176
  • 176. Business as Usual a b c physical delete Linked Lists 177
  • 177. Business as Usual a b Linked Lists c 178
  • 178. Validation private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Linked Lists 179
  • 179. List Validate Method private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Predecessor not Logically removed Linked Lists 180
  • 180. List Validate Method private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Current not Logically removed Linked Lists 181
  • 181. List Validate Method private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } Predecessor still Points to current Linked Lists 182
  • 182. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 183
  • 183. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; Validate as before } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 184
  • 184. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; Key found }}} finally { pred.unlock(); curr.unlock(); }}} Linked Lists 185
  • 185. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); Logical remove curr.unlock(); }}} Linked Lists 186
  • 186. Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); physical remove curr.unlock(); }}} Linked Lists 187
  • 187. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Linked Lists 188
  • 188. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Start at the head Linked Lists 189
  • 189. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Search key range Linked Lists 190
  • 190. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Traverse without locking (nodes may have been removed) Linked Lists 191
  • 191. Contains public boolean contains(Item item) { int key = item.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Present and undeleted? Linked Lists 192
  • 192. Summary: Wait-free Contains a 0 b 0 d 1 c0 e 0 Use Mark bit + list ordering 1. Not marked  in the set 2. Marked or missing  not in the set Linked Lists 193
  • 193. Lazy List a 0 b 0 d 1 c0 e 0 Lazy add() and remove() + Wait-free contains() Linked Lists 194
  • 194. Evaluation • Good: – – – – contains() doesn’t lock In fact, its wait-free! Good because typically high % contains() Uncontended calls don’t re-traverse • Bad – Contended add() and remove() calls do re-traverse – Traffic jam if one thread delays Linked Lists 195
  • 195. Traffic Jam • Any concurrent data structure based on mutual exclusion has a weakness • If one thread – Enters critical section – And “eats the big muffin” • Cache miss, page fault, descheduled … – Everyone else using that lock is stuck! – Need to trust the scheduler…. Linked Lists 196
  • 196. Reminder: Lock-Free Data Structures • No matter what … – Guarantees minimal progress in any execution – i.e. Some thread will always complete a method call – Even if others halt at malicious times – Implies that implementation can’t use locks Linked Lists 197
  • 197. Lock-free Lists • Next logical step – Wait-free contains() – lock-free add() and remove() • Use only compareAndSet() – What could go wrong? Linked Lists 198
  • 198. Remove Using CAS Logical Removal = Set Mark Bit a 0 b 0 Use CAS to verify pointer is correct Not enough! Linked Lists c 1 c0 e 0 Physical Removal CAS pointer 199
  • 199. Problem… Logical Removal = Set Mark Bit a 0 b 0 c 1 c0 Problem: Physical d not added to list… Removal Must Prevent CAS manipulation of removed node’s pointer Linked Lists e 0 d 0 Node added Before Physical Removal CAS 200
  • 200. The Solution: Combine Bit and Pointer Logical Removal = Set Mark Bit a 0 b 0 e 0 c 1 c0 d 0 Physical Removal Fail CAS: Node not added after logical CAS Removal Mark-Bit and Pointer are CASed together (AtomicMarkableReference) Linked Lists 201
  • 201. Solution • Use AtomicMarkableReference • Atomically – Swing reference and – Update flag • Remove in two steps – Set mark bit in next field – Redirect predecessor’s pointer Linked Lists 202
  • 202. Marking a Node • AtomicMarkableReference class – Java.util.concurrent.atomic package Reference address F mark bit Linked Lists 203
  • 203. Extracting Reference & Mark Public Object get(boolean[] marked); Linked Lists 204
  • 204. Extracting Reference & Mark Public Object get(boolean[] marked); Returns reference Returns mark at array index 0! Linked Lists 205
  • 205. Extracting Mark Only public boolean isMarked(); Value of mark Linked Lists 206
  • 206. Changing State Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); Linked Lists 207
  • 207. Changing State If this is the current reference … Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); And this is the current mark … Linked Lists 208
  • 208. Changing State …then change to this new reference … Public boolean compareAndSet( Object expectedRef, Object updateRef, boolean expectedMark, boolean updateMark); … and this new mark Linked Lists 209
  • 209. Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); Linked Lists 210
  • 210. Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); If this is the current reference … Linked Lists 211
  • 211. Changing State public boolean attemptMark( Object expectedRef, boolean updateMark); .. then change to this new mark. Linked Lists 212
  • 212. Removing a Node a CAS b c d remov e c Linked Lists 213
  • 213. Removing a Node failed CAS a CAS b c d remov e c remov e b Linked Lists 214
  • 214. Removing a Node a b c d remov e c remov e b Linked Lists 215
  • 215. Removing a Node a d remov e c remov e b Linked Lists 216
  • 216. Traversing the List • Q: what do you do when you find a “logically” deleted node in your path? • A: finish the job. – CAS the predecessor’s next field – Proceed (repeat as needed) Linked Lists 217
  • 217. Lock-Free Traversal (only Add and Remove) pred curr pred a CAS curr b c d Uh-oh Linked Lists 218
  • 218. The Window Class class Window public Node public Node Window(Node this.pred } } { pred; curr; pred, Node curr) { = pred; this.curr = curr; Linked Lists 219
  • 219. The Window Class class Window public Node public Node Window(Node this.pred } } { pred; curr; pred, Node curr) { = pred; this.curr = curr; A container for pred and current values Linked Lists 220
  • 220. Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Linked Lists 221
  • 221. Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Find returns window Linked Lists 222
  • 222. Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Extract pred and curr Linked Lists 223
  • 223. The Find Method Window window = find(item); item At some instant, pred curr or … succ Linked Lists© Herlihy-Shavit 2007 224
  • 224. The Find Method Window window = find(item); At some instant, pred curr= null item not in list succ Linked Lists© Herlihy-Shavit 2007 225
  • 225. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 226
  • 226. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists Keep trying 227
  • 227. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Find neighbors Linked Lists 228
  • 228. Remove public boolean remove(T item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} She’s not there … Linked Lists 229
  • 229. Remove public boolean remove(T item) { Boolean Try to mark node as deleted snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 230
  • 230. Remove public boolean remove(T item) { Boolean snip; If it doesn’t while just { work,(true) retry, Window window = find(head, key); if it does, job Node pred = window.pred, curr = window.curr; if (curr.key != essentially donekey) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 231
  • 231. Remove public boolean remove(T item) { Boolean snip; while (true) { a Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; Try to advance reference } else { (if succ = curr.next.getReference(); Node we don’t succeed, someone else did or will). snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Linked Lists 232
  • 232. Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Linked Lists 233
  • 233. Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Item already there. Linked Lists 234
  • 234. Add public boolean add(T item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} create new node Linked Lists 235
  • 235. Add public boolean add(T item) { boolean splice; Install new node, while (true) { else retry loop Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true;} }}} Linked Lists 236
  • 236. Wait-free Contains public boolean contains(Tt item) { boolean marked; int key = item.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) } Linked Lists 237
  • 237. Wait-free Contains public boolean contains(T item) { Only diff is that we boolean marked; get and check int key = item.hashCode(); Node curr = this.head; marked while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) } Linked Lists 238
  • 238. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 239
  • 239. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); If list changes while (true) { while succ = curr.next.get(marked); while (marked[0]) { traversed, … start over } Lock-Free if (curr.key >= key) because we return new Window(pred, curr); pred = curr; start over only curr = succ; if someone else } }} Linked Lists makes progress 240
  • 240. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr looking succ = head Start = null, from null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 241
  • 241. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { Move down the list pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 242
  • 242. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = Get ref to successor succ; } }} and current deleted bit Linked Lists 243
  • 243. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } Try }} if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; to remove deleted nodes in } path…code details soon Linked Lists 244
  • 244. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); Ifwhile (true)that is greater or curr key { succ curr.next.get(marked); equal,= return pred and curr while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 245
  • 245. Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); Otherwise advance window and while (marked[0]) { … loop again } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Linked Lists 246
  • 246. Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 247
  • 247. Lock-free Find Try to snip out node retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 248
  • 248. Lock-free Find if predecessor’s next field changed must retry whole traversal { retry: while (true) … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 249
  • 249. Lock-free Find Otherwise move on to check if next node deleted retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Linked Lists 250
  • 250. Performance On 16 node shared memory machine Benchmark throughput of Java List-based Set algs. Vary % of Contains() method Calls. Linked Lists 251
  • 251. High Contains Ratio Lock-free Lazy list Course Grained Fine Lock-coupling Linked Lists 252
  • 252. Low Contains Ratio Lock-free Lazy list Course Grained Fine Lock-coupling Linked Lists 253
  • 253. As Contains Ratio Increases Lock-free Lazy list Course Grained Fine Lock-coupling % Contains() Linked Lists 254
  • 254. Summary • • • • Coarse-grained locking Fine-grained locking Optimistic synchronization Lock-free synchronization Linked Lists 255