1. The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming 第 5 章 by Maurice Herlihy & Nir Shavit 日本語追記 by 根本和郎

6. From Weakest Register SRSW の最もシンプルなケース 1 0 1 Single reader Single writer Safe Boolean register

7. All the way to a Wait-free Implementation of Atomic Snapshots MRMW MRSW SRSW Saf e Regular Atomic M-valued Boolean Snapshot

9. Why is Mutual Exclusion so wrong? シンクロナイズドの排他制御のよくないところは・・・ (2)

10. Asynchronous Interrupts 同期的だと一人に休まれると、一緒に働いている人も休まなくてはならない ??? ??? (2) Swapped out back at

11. Heterogeneous Processors 直接つながっている場合遅い性能の処理に待たされることがある。 ??? ??? Yawn ( ひま ) (1) Pentium Pentium 286

12. Fault-tolerance 一人のダウンが致命的 ??? ??? (2)

15. FIFO Queue: Enqueue Method q.enq ( )

16. FIFO Queue: Dequeue Method q.deq()/

17. Two-Thread Wait-Free Queue public class LockFreeQueue { int head = 0, tail = 0; Item[QSIZE] items; public void enq(Item x) { while (tail - head == QSIZE) {}; items[tail % QSIZE] = x; tail++; } public Item deq() { while (tail - head == 0) {} Item item = items[head % QSIZE]; head++; return item; }}

18. What About Multiple Dequeuers?

22. Consensus: Each Thread has a Private Input 32 19 21

23. They Communicate

24. They Agree on One Thread’s Input 19 19 19

32. The Two-Move Tree Initial state この時は 1/0 の 2 価の共有オブジェクト Final states (2) A moves B moves

34. Bivalent: Both Possible 1 1 1 Bivalent(2 価 ) :2 種類の状態を取りうる 1 0 0 本ではここが 1 だが、それはそれで正しい図

35. Univalent: Single Value Possible 1 1 1 univalent ここから下は 1 価に固定される 1 0 0

36. x-valent: x Only Possible Decision 0 1 1 1 1-valent 0 1

45. What if inputs differ? (2) 1 0 By Way of contradiction ( 否定 ) : If univalent all executions must decide on same value

49. Critical States 次のステップで状態が絞り込まれていくステートのこと 0-valent 1-valent critical (3) (3)

50. From a Critical State A か B の決定が先勝ちで決まる状態 c If A goes first, protocol decides 0 If B goes first, protocol decides 1 0-valent 1-valent

51. Reaching Critical State 相手側が決定しないうちはまだ univalent これはだいたいの概念図 C A C A C B c C B univalent univalent univalent univalent 0-valent 1-valent initially bivalent

56. Possible Interactions ? ? ? ? y.write() ? ? ? ? x.write() ? ? ? ? y.read() ? ? ? ? x.read() y.write() x.write() y.read() x.read() A reads x A reads y

57. Some Thread Reads A runs solo, eventually decides 0 B reads x 1 0 A runs solo, eventually decides 1 c States look the same to A Contradiction

58. Possible Interactions ? ? no no y.write() ? ? no no x.write() no no no no y.read() no no no no x.read() y.write() x.write() y.read() x.read()

59. Writing Distinct Registers A writes y B writes x 1 0 c The song is the same A writes y B writes x Contradiction

60. Possible Interactions ? no no no y.write() no ? no no x.write() no no no no y.read() no no no no x.read() y.write() x.write() y.read() x.read()

61. Writing Same Registers States look the same to A A writes x B writes x 1 A runs solo, eventually decides 1 c 0 A runs solo, eventually decides 0 A writes x Contradiction

62. That’s All, Folks! QED no no no no y.write() no no no no x.write() no no no no y.read() no no no no x.read() y.write() x.write() y.read() x.read()

64. What Does Consensus have to do with Concurrent Objects?

65. Consensus Object public interface Consensus { Object decide(object value); } (4) (4)

68. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4)

69. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4) Each thread’s proposed value

70. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4) Propose a value

71. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4) Decide a value: abstract method means subclass does the heavy lifting (real work)

72. Can a FIFO Queue Implement Consensus?

73. FIFO Consensus proposed array FIFO Queue with red and black balls Coveted red ball Dreaded black ball 8

74. Protocol: Write Value to Array 0 1 0

75. Protocol: Take Next Item from Queue 0 0 1 8

77. Consensus Using FIFO Queue public class QueueConsensus extends ConsensusProtocol { private Queue queue; public QueueConsensus() { queue = new Queue(); queue.enq(Ball.RED); queue.enq(Ball.BLACK); } … }

78. Initialize Queue public class QueueConsensus extends ConsensusProtocol { private Queue queue; public QueueConsensus() { this .queue = new Queue(); this .queue.enq(Ball.RED); this .queue.enq(Ball.BLACK); } … } 8

79. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this .queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } }

80. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this .queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-ij]; } } Race to dequeue first queue item

81. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } } i = ThreadID.get(); I win if I was first

82. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } } Other thread wins if I was second

95. Double Assignment Interface interface Assign2 { public void assign( int i 1 , int v 1 , int i 2 , int v 2 ); public int read( int i); } (4) (4)

96. Double Assignment Interface interface Assign2 { public void assign( int i 1 , int v 1 , int i 2 , int v 2 ); public int read(int i); } (4) (4) Atomically assign value[i 1 ]= v 1 value[i 2 ]= v 2

97. Double Assignment Interface interface Assign2 { public void assign(int i 1 , int v 1 , int i 2 , int v 2 ); public int read( int i); } (4) (4) Return i-th value

99. Thread A wins if A B (1) Thread B didn’t move

100. Thread A wins if A B (1) Thread B moved later

101. Thread A loses if A B (1) Thread B moved earlier 先に書いた方の勝ち

102. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; ←j が突然出てくる ? }} (4) (4)

103. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Extends ConsensusProtocol Decide sets j=i-1 and proposes value

104. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Three slots initialized to EMPTY

105. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Assign id 0 to entries 0,1 (or id 1 to entries 1,2)

106. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Read the register my thread didn’t assign

107. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY ||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Other thread didn’t move, so I win

108. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY|| other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Other thread moved later so I win

109. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) OK, I win.

110. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Other thread moved first, so I lose (1)

113. Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = this. value; this. value = mumble( this. value); return prior; } } (1)

114. Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = this. value; this.value = mumble(this.value); return prior; } } (1) Return prior value

115. Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = this.value; this. value = mumble( this. value); return prior; } } (1) Apply function to current value

117. Example: Read public abstract class RMWRegister { private int value; public int synchronized read() { int prior = this. value; this. value = this. value; return prior; } } (1)

118. Example: Read public abstract class RMW { private int value; public int synchronized read() { int prior = this.value; this. value = this. value; return prior; } } (1) Apply f(v)=v , the identity function

119. Example: getAndSet public abstract class RMWRegister { private int value; public int synchronized getAndSet( int v) { int prior = this. value; this. value = v; return prior; } … } (1)

120. Example: getAndSet (swap) public abstract class RMWRegister { private int value; public int synchronized getAndSet(int v) { int prior = this.value; this. value = v; return prior; } … } (1) F(x)=v is constant function

121. getAndIncrement public abstract class RMWRegister { private int value; public int synchronized getAndIncrement() { int prior = this. value; this. value = this .value + 1; return prior; } … } (1)

122. getAndIncrement public abstract class RMWRegister { private int value; public int synchronized getAndIncrement() { int prior = this.value; this. value = this .value + 1; return prior; } … } (1) F(x) = x+1

123. getAndAdd public abstract class RMWRegister { private int value; public int synchronized getAndAdd( int a) { int prior = this. value; this. value = this .value + a; return prior; } … } (1)

124. Example: getAndAdd public abstract class RMWRegister { private int value; public int synchronized getAndIncrement(int a) { int prior = this.value; this. value = this .value + a; return prior; } … } (1) F(x) = x+a

125. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet( int expected, int update) { int prior = this. value; if ( this .value==expected) { this .value = update; return true ; } return false ; } … } (1)

126. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if ( this .value==expected ) { this.value = update; return true; } return false; } … } (1) If value is what was expected, …

127. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update ) { int prior = this.value; if (this.value==expected) { this .value = update; return true; } return false; } … } (1) … replace it

128. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if (this.value==expected) { this.value = update; return true ; } return false; } … } (1) Report success

129. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if (this.value==expected) { this.value = update; return true; } return false ; } … } (1) Otherwise report failure

130. Read-Modify-Write public abstract class RMWRegister { private int value; public void synchronized getAndMumble() { int prior = this.value; this.value = mumble( this. value); return prior; } } (1) Lets characterize F(x)…

135. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4)

136. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) Initialized to v

137. Proof public class RMWConsensus extends Consensus { private RMWRegister r = v; public Object decide(object value) { if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) Am I first?

138. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) Yes, return my input

139. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) No, return other’s input

143. Meanwhile Back at the Critical State c 0-valent 1-valent A about to apply f A B about to apply f B

144. Maybe the Functions Commute c 0-valent A applies f A B applies f B A applies f A B applies f B 0 1 C runs solo C runs solo 1-valent

145. Maybe the Functions Commute c 0-valent A applies f A B applies f B A applies f A B applies f B 0 1 C runs solo C runs solo 1-valent These states look the same to C Contradiction

146. Maybe the Functions Overwrite c 0-valent A applies f A B applies f B A applies f A 0 1 C runs solo C runs solo 1-valent

147. Maybe the Functions Overwrite c 0-valent A applies f A B applies f B A applies f A 0 1 C runs solo C runs solo 1-valent These states look the same to C Contradiction

149. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet( int expected, int update) { int prior = this. value; if ( this .value==expected) { this .value = update; return true ; } return false ; } … } (1)

150. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if ( this .value==expected ) { this.value = update; return true; } return false; } … } (1) replace value if its what we expected, …

151. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } } (4)

152. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value) r.compareAndSet(-1,i); return proposed[r.get()]; } } (4) Initialized to -1

153. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } } (4) Try to swap in my id

154. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } } (4) Decide winner’s preference

155. The Consensus Hierarchy 1 Read/Write Registers, Snapshots… 2 getAndSet, getAndIncrement, … ∞ compareAndSet,… . . .

167. Java.util.concurrent による並行制御 Ｔｈｒｅａｄ－１ Ｔｈｒｅａｄ－ 2 Ｔｈｒｅａｄ－ 3 Abstract Synchronized Queue Abstract Synchronized Queue Abstract Synchronized Queue 更新先 先に完成したものを準備しておいて 一瞬で cas で切り替えることで Lock をかけない 各スレッドからの要求を Queue で受けて、待たせない