Successfully reported this slideshow.
The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming  第 5 章  by Mau...
並行プログラミングの原理から実践まで <ul><li>この powerpoint は 「並行プログラミングの原理から実践まで」第 5 章を読むための補助教材です。 </li></ul><ul><li>本とはカバー範囲が異なります。またコードサン...
<ul><li>Mathematical model of  concurrent  computation </li></ul><ul><li>What is (and is not) concurrently computable </li...
Wait-Free Implementation <ul><li>Every method call completes in finite number of steps </li></ul><ul><li>Implies no mutual...
コンセンサス・プロトコル <ul><li>コンセンサスオブジェクト </li></ul><ul><ul><li>public interface Consensus<T> { </li></ul></ul><ul><ul><li>T decid...
From Weakest Register SRSW の最もシンプルなケース 1 0 1 Single reader Single writer Safe Boolean register
All the way to a Wait-free Implementation of Atomic Snapshots MRMW MRSW SRSW Saf e Regular Atomic M-valued Boolean Snapshot
Rationale for wait-freedom <ul><li>We wanted atomic registers to implement mutual exclusion </li></ul><ul><li>So we couldn...
Why is Mutual Exclusion so wrong? シンクロナイズドの排他制御のよくないところは・・・ (2)
Asynchronous Interrupts 同期的だと一人に休まれると、一緒に働いている人も休まなくてはならない ??? ??? (2) Swapped out back at
Heterogeneous  Processors 直接つながっている場合遅い性能の処理に待たされることがある。 ??? ??? Yawn ( ひま ) (1) Pentium Pentium 286
Fault-tolerance 一人のダウンが致命的 ??? ??? (2)
Machine Level Instruction Granularity Eugene Amdahl (2) <ul><li>マシンごとでアクセスする粒度が一致しないかもしれない </li></ul>
Basic Questions <ul><li>Wait-Free  synchronization might be a good idea in principle </li></ul><ul><li>But how do you do i...
FIFO Queue: Enqueue Method q.enq ( )
FIFO Queue: Dequeue Method q.deq()/
Two-Thread Wait-Free Queue public class  LockFreeQueue { int  head = 0, tail = 0;  Item[QSIZE] items; public void  enq(Ite...
What About Multiple Dequeuers?
Grand Challenge <ul><li>Implement a FIFO queue </li></ul><ul><ul><li>Wait-free   (FIFO は再試行がない ) </li></ul></ul><ul><ul><l...
Consensus <ul><li>While you are ruminating ( 反復思考 )  on the grand challenge… </li></ul><ul><li>We will give you another pu...
コンセンサス問題 <ul><li>分散システム上の全プロセス(または全正常プロセス)が同じ値( 0 または 1 )で合意する問題 </li></ul><ul><ul><li>複数プロセスである処理を行っているとき、全体の状況(進捗状態、分岐条件...
Consensus: Each Thread has a Private Input 32 19 21
They Communicate
They Agree on One Thread’s Input 19 19 19
Formally: Consensus <ul><li>Consistent : all threads decide the same value </li></ul><ul><li>Valid : the common decision v...
No Wait-Free Implementation of Consensus using Registers ??? ??? <ul><li>wait-Free な結合だと、障害時に同意形成できない </li></ul>
Formally <ul><li>Theorem [adapted from Fischer, Lynch, Paterson]:  There is no wait-free implementation of n-thread consen...
Proof Strategy <ul><li>Assume otherwise </li></ul><ul><li>Reason about the properties of any such protocol </li></ul><ul><...
Wait-Free Computation <ul><li>Either A or  B  “moves” </li></ul><ul><li>Moving means </li></ul><ul><ul><li>Register read <...
A スレッドと B スレッドの話 <ul><li>A スレッドと B スレッドが 2 回ずつ動くモデルで考える。 </li></ul>
バイナリコンセンサス <ul><li>次ページで語ろうとしているのはバイナリコンセンサス、 </li></ul><ul><li>つまり  1/0  判定の最も基本的意思決定 </li></ul><ul><li>登場スレッドは A と B のふた...
The Two-Move Tree Initial state この時は 1/0 の 2 価の共有オブジェクト Final states (2) A moves B moves
Decision Values コンセンサス数の決め方 すべてのスレッドが同じ値を共有するまで 不定 どちらかに確定 不定 0 に確定 どちらかに確定 0 に確定 1 0 0 1 1 1 不定 1 回目 2 回目 3 回目 4 回目 1 と読む...
Bivalent: Both Possible 1 1 1 Bivalent(2 価 ) :2 種類の状態を取りうる 1 0 0 本ではここが 1 だが、それはそれで正しい図
Univalent: Single Value Possible 1 1 1 univalent ここから下は 1 価に固定される 1 0 0
x-valent: x Only Possible Decision 0 1 1 1 1-valent 0 1
 
Summary <ul><li>Wait-free computation is a tree </li></ul><ul><li>Bivalent system states </li></ul><ul><ul><li>Outcome not...
まとめ <ul><li>下記の順で状態遷移していく </li></ul><ul><ul><li>Bivalent 不定 </li></ul></ul><ul><ul><li>Univalent 確定しているが未観測 </li></ul></ul...
Claim <ul><li>Some initial state is bivalent </li></ul><ul><li>Outcome depends on </li></ul><ul><ul><li>Chance </li></ul><...
Both Inputs 0 <ul><li>Univalent: all executions must decide  0 </li></ul>0 0 (2) (1)
Both Inputs 0 <ul><li>Including this solo execution by  A </li></ul>(1) (1) 0
Both Inputs 1 <ul><li>All executions must decide  1 </li></ul>1 1 (2) (1)
Both Inputs 1 <ul><li>Including this solo execution by  B </li></ul>(1) (1) 1
What if inputs differ? (2) 1 0 By Way of contradiction ( 否定 ) :  If univalent all executions must decide on same value
The Possible Executions <ul><li>Include the solo execution by A </li></ul><ul><li>that decides  0 </li></ul>0 1 (2)
The Possible Executions <ul><li>Also include the solo execution by  B </li></ul><ul><li>which we know decides  1 </li></ul...
Possible Executions Include <ul><li>Solo execution by A must decide  0 </li></ul><ul><li>Solo execution by  B  must decide...
Critical States 次のステップで状態が絞り込まれていくステートのこと 0-valent 1-valent critical (3) (3)
From a Critical State A か B の決定が先勝ちで決まる状態 c If A goes first, protocol decides 0 If B goes first, protocol decides 1 0-vale...
Reaching Critical State 相手側が決定しないうちはまだ univalent これはだいたいの概念図 C A C A C B c C B univalent univalent univalent univalent 0-v...
Critical States <ul><li>Starting from a bivalent initial state </li></ul><ul><li>The protocol can reach a critical state <...
Model Dependency <ul><li>So far, memory-independent! </li></ul><ul><li>True for </li></ul><ul><ul><li>Registers </li></ul>...
Read-Write Memory <ul><li>Reads and/or writes </li></ul><ul><li>To same/different registers </li></ul>
Completing the Proof <ul><li>Lets look at executions that: </li></ul><ul><ul><li>Start from a critical state </li></ul></u...
Possible Interactions ? ? ? ? y.write() ? ? ? ? x.write() ? ? ? ? y.read() ? ? ? ? x.read() y.write() x.write() y.read() x...
Some Thread Reads A runs solo, eventually decides 0 B reads x 1 0 A runs solo, eventually  decides 1 c States look the sam...
Possible Interactions ? ? no no y.write() ? ? no no x.write() no no no no y.read() no no no no x.read() y.write() x.write(...
Writing Distinct Registers A writes y B writes x 1 0 c The song is the same A writes y B writes x Contradiction
Possible Interactions ? no no no y.write() no ? no no x.write() no no no no y.read() no no no no x.read() y.write() x.writ...
Writing Same Registers States look the same to A A writes x B writes x 1 A runs solo, eventually decides 1 c 0 A runs solo...
That’s All, Folks! QED no no no no y.write() no no no no x.write() no no no no y.read() no no no no x.read() y.write() x.w...
Recap: Atomic Registers Can’t Do Consensus  アトミックなレジスターのみではまだ足りない <ul><li>If protocol exists </li></ul><ul><ul><li>It has ...
What Does Consensus have to do with Concurrent Objects?
Consensus Object public interface  Consensus { Object decide(object value); } (4) (4)
Concurrent Consensus Object <ul><li>We consider only one time objects: each thread can execute a method only once </li></u...
Java Jargon Watch <ul><li>Define  Consensus protocol  as an  abstract class </li></ul><ul><li>We implement some methods </...
Generic Consensus Protocol abstract class  ConsensusProtocol  implements  Consensus { protected  Object[] proposed = new  ...
Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected  Object[] proposed = new  Obj...
Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Objec...
Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Objec...
Can a FIFO Queue Implement Consensus?
FIFO Consensus proposed  array FIFO Queue  with red and black balls Coveted red ball Dreaded black ball 8
Protocol: Write Value to Array 0 1 0
Protocol: Take Next Item from Queue 0 0 1 8
Protocol: Take Next Item from Queue 0 1 I got the coveted( 熱望 ) red ball, so I will decide my value I got the dreaded( 恐れる...
Consensus Using FIFO Queue public class  QueueConsensus extends  ConsensusProtocol { private Queue queue; public  QueueCon...
Initialize Queue public class QueueConsensus extends ConsensusProtocol { private Queue queue; public QueueConsensus() { th...
Who Won? public class  QueueConsensus extends  ConsensusProtocol { private  Queue queue; … public  decide(object value) { ...
Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { prop...
Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { prop...
Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { prop...
Why does this Work? <ul><li>If one thread gets the  red  ball </li></ul><ul><li>Then the other gets the  black  ball </li>...
Theorem <ul><li>We can solve 2-thread consensus using only </li></ul><ul><ul><li>A two-dequeuer queue, and </li></ul></ul>...
Implications( 関わり合い ) <ul><li>Given </li></ul><ul><ul><li>A consensus protocol from queue and registers </li></ul></ul><ul...
Corollary( 結論 ) <ul><li>It is impossible to implement  </li></ul><ul><ul><li>a two-dequeuer wait-free FIFO queue </li></ul...
Consensus Numbers <ul><li>An object X has  consensus number   n </li></ul><ul><ul><li>If it can be used to solve  n -threa...
Consensus Numbers <ul><li>Theorem </li></ul><ul><ul><li>Atomic read/write registers have consensus number  1 </li></ul></u...
Consensus Numbers Measure Synchronization Power <ul><li>Theorem </li></ul><ul><ul><li>If  you can implement  X  from  Y </...
Synchronization Speed Limit <ul><li>Conversely </li></ul><ul><ul><li>If  X  has consensus number  c </li></ul></ul><ul><ul...
Earlier Grand Challenge <ul><li>Snapshot means </li></ul><ul><ul><li>Write any array element </li></ul></ul><ul><ul><li>Re...
Multiple Assignment Theorem <ul><li>Atomic registers cannot implement multiple assignment </li></ul><ul><li>Weird or what?...
Proof Strategy  不明 <ul><li>If we can write to 2/3 array elements </li></ul><ul><ul><li>We can solve 2-consensus </li></ul>...
Proof Strategy <ul><li>Take a  3 -element array </li></ul><ul><ul><li>A  writes atomically to slots  0  and  1 </li></ul><...
Double Assignment Interface interface  Assign2 { public void  assign( int  i 1 ,  int  v 1 , int  i 2 ,  int  v 2 ); publi...
Double Assignment Interface interface Assign2 { public void  assign( int  i 1 ,  int  v 1 , int  i 2 ,  int  v 2 ); public...
Double Assignment Interface interface Assign2 { public void assign(int i 1 , int v 1 , int i 2 , int v 2 ); public int  re...
Initially Writes to 0 and 1 Writes to 1 and 2 A B <ul><li>一つのスレッドが二つ以上を書く場合の競合の解説 </li></ul>
Thread  A  wins if A B (1) Thread B didn’t move
Thread  A  wins if A B (1) Thread B moved later
Thread  A  loses if A B (1) Thread B moved earlier 先に書いた方の勝ち
Multi-Consensus Code class  MultiConsensus  extends  …{ Assign2 a =  new  Assign2(3, EMPTY); public  Object decide(object ...
Multi-Consensus Code class  MultiConsensus  extends  …{ Assign2 a =  new  Assign2(3, EMPTY); public  Object decide(object ...
Multi-Consensus Code class MultiConsensus extends …{ Assign2 a =  new  Assign2(3, EMPTY); public Object decide(object valu...
Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value)...
Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value)...
Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value)...
Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value)...
Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value)...
Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value)...
Summary <ul><li>If a thread can assign atomically to  2  out of  3  array locations </li></ul><ul><li>Then we can solve  2...
Read-Modify-Write Objects <ul><li>Method call </li></ul><ul><ul><li>Returns object’s prior value  x </li></ul></ul><ul><ul...
Read-Modify-Write public abstract class  RMWRegister { private int  value; public int synchronized  getAndMumble() { int  ...
Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized  getAndMumble() { int  pr...
Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized  getAndMumble() { int pri...
RMW Everywhere! <ul><li>Most synchronization instructions </li></ul><ul><ul><li>are RMW methods </li></ul></ul><ul><li>The...
Example: Read public abstract class  RMWRegister { private int  value; public int synchronized  read() { int  prior  =  th...
Example: Read public abstract class RMW { private int value; public int synchronized read() { int prior  = this.value; thi...
Example: getAndSet public abstract class  RMWRegister { private int  value; public int synchronized getAndSet( int  v) { i...
Example: getAndSet (swap) public abstract class RMWRegister { private int value; public int synchronized getAndSet(int v) ...
getAndIncrement public abstract class  RMWRegister { private int  value; public int synchronized getAndIncrement() { int  ...
getAndIncrement public abstract class RMWRegister { private int value; public int synchronized getAndIncrement() { int pri...
getAndAdd public abstract class  RMWRegister { private int  value; public int synchronized getAndAdd( int  a) { int  prior...
Example: getAndAdd public abstract class RMWRegister { private int value; public int synchronized getAndIncrement(int a) {...
compareAndSet public abstract class  RMWRegister { private int  value; public boolean synchronized compareAndSet( int  exp...
compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int  expect...
compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expecte...
compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expecte...
compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expecte...
Read-Modify-Write public abstract class RMWRegister { private int value; public void synchronized   getAndMumble() { int p...
Definition <ul><li>A RMW method </li></ul><ul><ul><li>With function  mumble(x) </li></ul></ul><ul><ul><li>is  non-trivial ...
Par Example <ul><li>Identity(x) = x </li></ul><ul><ul><li>is trivial  </li></ul></ul><ul><li>getAndIncrement(x) = x+1 </li...
Theorem <ul><li>Any non-trivial RMW object has consensus number at least  2 </li></ul><ul><li>No wait-free implementation ...
Reminder <ul><li>Subclasses of  consensus   have </li></ul><ul><ul><li>propose(x)   method </li></ul></ul><ul><ul><ul><li>...
Proof public class  RMWConsensus extends  ConsensusProtocol { private  RMWRegister r = v; public   Object  decide(object v...
Proof public class RMWConsensus extends ConsensusProtocol { private  RMWRegister r = v; public Object decide(object value)...
Proof public class RMWConsensus extends Consensus { private RMWRegister r = v; public Object decide(object value) { if  (r...
Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) ...
Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) ...
Proof <ul><li>We have displayed </li></ul><ul><ul><li>A two-thread consensus protocol </li></ul></ul><ul><ul><li>Using any...
Interfering RMW <ul><li>Let  F  be a set of functions such that for all  f i  and  f j ,  either </li></ul><ul><ul><li>Com...
Examples <ul><li>“ test-and-set”  getAndSet(1)  f(v)=1 </li></ul><ul><li>“ swap”  getAndSet(x)  f(v,x)=x </li></ul><ul><li...
Meanwhile Back at the Critical State c 0-valent 1-valent A about to apply f A B about to apply f B
Maybe the Functions Commute c 0-valent A applies f A B applies f B A applies f A B applies f B 0 1 C runs solo C runs solo...
Maybe the Functions Commute c 0-valent A applies f A B applies f B A applies f A B applies f B 0 1 C runs solo C runs solo...
Maybe the Functions Overwrite c 0-valent A applies f A B applies f B A applies f A 0 1 C runs solo C runs solo 1-valent
Maybe the Functions Overwrite c 0-valent A applies f A B applies f B A applies f A 0 1 C runs solo C runs solo 1-valent Th...
Impact <ul><li>Many early machines provided these “weak” RMW instructions </li></ul><ul><ul><li>Test-and-set (IBM 360) </l...
compareAndSet public abstract class  RMWRegister { private int  value; public boolean synchronized compareAndSet( int  exp...
compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int  expect...
compareAndSet Has ∞ Consensus Number public class  RMWConsensus extends  ConsensusProtocol { private  AtomicInteger r = ne...
compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private  AtomicInteger r = new ...
compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new A...
compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new A...
The Consensus Hierarchy 1 Read/Write Registers, Snapshots… 2 getAndSet, getAndIncrement, … ∞   compareAndSet,… . . .
Multiple Assignment <ul><li>Atomic  k -assignment </li></ul><ul><li>Solves consensus for  2k-2  threads </li></ul><ul><li>...
Lock-Freedom <ul><li>Lock-free:  in an infinite execution infinitely often some method call finishes  (obviously, in a fin...
Lock-free と Wait-free <ul><li>Lock-free と Wait-free アルゴリズム とは、共有データにロックをかけて保護するアルゴリズムとは違い、複数のスレッドが同 時並行的に、ある対象データを壊すことなしに読...
Lock-free への書き換え <ul><li>例えば、銀行口座への預金プログラムを作るとする。それぞれのスレッドを ATM とする。預金預け入れをするには、現在の預金残高を読み出し、預入金額を足し算し、新しい預金残高を書き込むという処理が必...
Lock-free な口座入金 <ul><li>Wait-free な queue が ATM 台数分ある。 </li></ul><ul><li>入金処理はそれぞれの queue が wait-free で受け付ける。 </li></ul><u...
Lock-Free vs. Wait-free <ul><li>Wait-Free:  each method call takes a finite number of steps to finish </li></ul><ul><ul><l...
Lock-Freedom <ul><li>Any  wait-free  implementation is  lock-free .  </li></ul><ul><li>Lock-free  is the same as wait-free...
Lock-Freedom <ul><li>wait-free   ならそれは  lock-free .  </li></ul><ul><li>Lock-free   は  wait-free と等価 . </li></ul><ul><li>すべ...
Lock-Free Implementations <ul><li>Lock-free consensus is as impossible as wait-free consensus  </li></ul><ul><li>All the r...
There is More: Universality <ul><li>Consensus is  universal </li></ul><ul><li>From  n -thread consensus </li></ul><ul><ul>...
Java.util.concurrent を使おう <ul><li>++ の操作は atomic ではないので危険 </li></ul>int counter = 0;  … . counter++;  import java.util.con...
Java.util.concurrent による並行制御 Thread-1 Thread- 2 Thread- 3 Abstract Synchronized Queue Abstract Synchronized Queue Abstract...
              This work is licensed under a  Creative Commons Attribution- ShareAlike  2.5 License .  <ul><li>You are free...
Upcoming SlideShare
Loading in …5
×

The Art of Multiprocessor Programmin Chapter 05 with Japanese comment.

17,239 views

Published on

「並行プログラミングの原理から実践まで」
日本語解説追記版。

Published in: Technology
  • Be the first to comment

The Art of Multiprocessor Programmin Chapter 05 with Japanese comment.

  1. 1. The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming 第 5 章  by Maurice Herlihy & Nir Shavit 日本語追記 by 根本和郎
  2. 2. 並行プログラミングの原理から実践まで <ul><li>この powerpoint は 「並行プログラミングの原理から実践まで」第 5 章を読むための補助教材です。 </li></ul><ul><li>本とはカバー範囲が異なります。またコードサンプルや図も本とは微妙に異なっているので併読時は注意しましょう。 </li></ul>
  3. 3. <ul><li>Mathematical model of concurrent computation </li></ul><ul><li>What is (and is not) concurrently computable </li></ul><ul><li>Efficiency (mostly) irrelevant </li></ul>Last Lecture: Shared-Memory Computability 10011 Shared Memory
  4. 4. Wait-Free Implementation <ul><li>Every method call completes in finite number of steps </li></ul><ul><li>Implies no mutual exclusion </li></ul><ul><li>各メソッド呼び出しが有限数のステップで完了する場合、その平行オブジェクト実装は無待機である。 </li></ul>(2)
  5. 5. コンセンサス・プロトコル <ul><li>コンセンサスオブジェクト </li></ul><ul><ul><li>public interface Consensus<T> { </li></ul></ul><ul><ul><li>T decide(T, value); </li></ul></ul><ul><ul><li>} </li></ul></ul><ul><li>参加するオブジェクトは皆、 decide() という判定メソッドを持つ。 </li></ul>
  6. 6. From Weakest Register SRSW の最もシンプルなケース 1 0 1 Single reader Single writer Safe Boolean register
  7. 7. All the way to a Wait-free Implementation of Atomic Snapshots MRMW MRSW SRSW Saf e Regular Atomic M-valued Boolean Snapshot
  8. 8. Rationale for wait-freedom <ul><li>We wanted atomic registers to implement mutual exclusion </li></ul><ul><li>So we couldn’t use mutual exclusion to implement atomic registers </li></ul><ul><li>But wait, there’s more! </li></ul><ul><li>選択排他で動作可能な、アトミックなレジスターがあれば良いのだが・・・ </li></ul><ul><li>しかしそれがなくても、単にメソッドを synchronized すること以上の Lock-free な何かがあるのではないか。 </li></ul>
  9. 9. Why is Mutual Exclusion so wrong? シンクロナイズドの排他制御のよくないところは・・・ (2)
  10. 10. Asynchronous Interrupts 同期的だと一人に休まれると、一緒に働いている人も休まなくてはならない ??? ??? (2) Swapped out back at
  11. 11. Heterogeneous Processors 直接つながっている場合遅い性能の処理に待たされることがある。 ??? ??? Yawn ( ひま ) (1) Pentium Pentium 286
  12. 12. Fault-tolerance 一人のダウンが致命的 ??? ??? (2)
  13. 13. Machine Level Instruction Granularity Eugene Amdahl (2) <ul><li>マシンごとでアクセスする粒度が一致しないかもしれない </li></ul>
  14. 14. Basic Questions <ul><li>Wait-Free synchronization might be a good idea in principle </li></ul><ul><li>But how do you do it </li></ul><ul><ul><li>Systematically? </li></ul></ul><ul><ul><li>Correctly? </li></ul></ul><ul><ul><li>Efficiently? </li></ul></ul><ul><li>wait-Free な同期は、ここでは java.util.Concurrent を使いこなしましょう、という話しへの導入 </li></ul>
  15. 15. FIFO Queue: Enqueue Method q.enq ( )
  16. 16. FIFO Queue: Dequeue Method q.deq()/
  17. 17. Two-Thread Wait-Free Queue public class LockFreeQueue { int head = 0, tail = 0; Item[QSIZE] items; public void enq(Item x) { while (tail - head == QSIZE) {}; items[tail % QSIZE] = x; tail++; } public Item deq() { while (tail - head == 0) {} Item item = items[head % QSIZE]; head++; return item; }}
  18. 18. What About Multiple Dequeuers?
  19. 19. Grand Challenge <ul><li>Implement a FIFO queue </li></ul><ul><ul><li>Wait-free   (FIFO は再試行がない ) </li></ul></ul><ul><ul><li>Linearizable </li></ul></ul><ul><ul><li>From atomic read-write registers </li></ul></ul><ul><ul><li>Multiple dequeuers </li></ul></ul>Only new aspect (1)
  20. 20. Consensus <ul><li>While you are ruminating ( 反復思考 ) on the grand challenge… </li></ul><ul><li>We will give you another puzzle </li></ul><ul><ul><li>Consensus </li></ul></ul><ul><ul><li>Will be important … </li></ul></ul>
  21. 21. コンセンサス問題 <ul><li>分散システム上の全プロセス(または全正常プロセス)が同じ値( 0 または 1 )で合意する問題 </li></ul><ul><ul><li>複数プロセスである処理を行っているとき、全体の状況(進捗状態、分岐条件、終了条件)に関して全プロセスで合意が必要な場合あり </li></ul></ul><ul><ul><li>信頼性のための多重化システムでは、各サブシステムでの出力の合意として最終出力を決定 </li></ul></ul>
  22. 22. Consensus: Each Thread has a Private Input 32 19 21
  23. 23. They Communicate
  24. 24. They Agree on One Thread’s Input 19 19 19
  25. 25. Formally: Consensus <ul><li>Consistent : all threads decide the same value </li></ul><ul><li>Valid : the common decision value is some thread's input </li></ul><ul><li>整合性がある : 全員が一致 </li></ul><ul><li>有効 : 多数決的に正しい </li></ul>
  26. 26. No Wait-Free Implementation of Consensus using Registers ??? ??? <ul><li>wait-Free な結合だと、障害時に同意形成できない </li></ul>
  27. 27. Formally <ul><li>Theorem [adapted from Fischer, Lynch, Paterson]: There is no wait-free implementation of n-thread consensus, n>1, from read-write registers </li></ul><ul><li>Implication: asynchronous computability fundamentally different from Turing computability </li></ul>PODC2001 の Influential paper award CAP 理論の元
  28. 28. Proof Strategy <ul><li>Assume otherwise </li></ul><ul><li>Reason about the properties of any such protocol </li></ul><ul><li>Derive a contradiction ( 矛盾を派生しうる ) </li></ul><ul><li>Quod Erat Demonstrandum </li></ul><ul><li>Suffices to prove for binary consensus and n=2 </li></ul><ul><li>バイナリコンセンサスとは、つまり並行処理を 2 分岐探索に置換して評価する確率分布 </li></ul><ul><li>この最小構成で検討してみる。 </li></ul>
  29. 29. Wait-Free Computation <ul><li>Either A or B “moves” </li></ul><ul><li>Moving means </li></ul><ul><ul><li>Register read </li></ul></ul><ul><ul><li>Register write </li></ul></ul><ul><li>操作 A/B がある、 A は読む B は書く </li></ul><ul><li>各スレッドは最大 1 回の意思決定をする、つまり decide() メソッド </li></ul>A moves 読む B moves 書く Lemma = 補題 (dilemma: ジレンマ )
  30. 30. A スレッドと B スレッドの話 <ul><li>A スレッドと B スレッドが 2 回ずつ動くモデルで考える。 </li></ul>
  31. 31. バイナリコンセンサス <ul><li>次ページで語ろうとしているのはバイナリコンセンサス、 </li></ul><ul><li>つまり 1/0 判定の最も基本的意思決定 </li></ul><ul><li>登場スレッドは A と B のふたつ </li></ul><ul><li>初期値は不定 (1 か 0) </li></ul><ul><li>並行コンセンサスを逐次コンセンサスに線形化して考える </li></ul><ul><li>初期状態として A は 0 を、 B は 1 を受け取ると仮定する </li></ul><ul><li>Read と Write がどのようにインターリーブされるかで決まる </li></ul><ul><li>論理的分岐ではなく確率分布図のようなものとして考える </li></ul>
  32. 32. The Two-Move Tree Initial state この時は 1/0 の 2 価の共有オブジェクト Final states (2) A moves B moves
  33. 33. Decision Values コンセンサス数の決め方 すべてのスレッドが同じ値を共有するまで 不定 どちらかに確定 不定 0 に確定 どちらかに確定 0 に確定 1 0 0 1 1 1 不定 1 回目 2 回目 3 回目 4 回目 1 と読む 0 と読む 1 と書く 1 と書く 1 と読む 1 と読む 1 と書く 1 と書く 1 と書く 読む 1 と書く 1 と書く 0 と書く 1 と書く 読む 0 と書く 読む 0 と書く <ul><li>下図を深く分析してはいけない、もっとざっくりしたもの。 </li></ul>
  34. 34. Bivalent: Both Possible 1 1 1 Bivalent(2 価 ) :2 種類の状態を取りうる 1 0 0 本ではここが 1 だが、それはそれで正しい図
  35. 35. Univalent: Single Value Possible 1 1 1 univalent ここから下は 1 価に固定される 1 0 0
  36. 36. x-valent: x Only Possible Decision 0 1 1 1 1-valent 0 1
  37. 38. Summary <ul><li>Wait-free computation is a tree </li></ul><ul><li>Bivalent system states </li></ul><ul><ul><li>Outcome not fixed </li></ul></ul><ul><li>Univalent states </li></ul><ul><ul><li>Outcome is fixed </li></ul></ul><ul><ul><li>May not be “known” yet </li></ul></ul><ul><li>1-Valent and 0-Valent states </li></ul><ul><li>Wait-free は木構造の計算モデル </li></ul><ul><li>Bivalent の状態は不定 </li></ul><ul><li>Univalent は固定した状態だが未観測 </li></ul>
  38. 39. まとめ <ul><li>下記の順で状態遷移していく </li></ul><ul><ul><li>Bivalent 不定 </li></ul></ul><ul><ul><li>Univalent 確定しているが未観測 </li></ul></ul><ul><ul><li>0-Valent ,1-Valent 確定 </li></ul></ul>
  39. 40. Claim <ul><li>Some initial state is bivalent </li></ul><ul><li>Outcome depends on </li></ul><ul><ul><li>Chance </li></ul></ul><ul><ul><li>Whim( 気まぐれ ) of the scheduler </li></ul></ul><ul><li>Multiprocessor gods do play dice … </li></ul><ul><li>Lets prove this claim </li></ul>
  40. 41. Both Inputs 0 <ul><li>Univalent: all executions must decide 0 </li></ul>0 0 (2) (1)
  41. 42. Both Inputs 0 <ul><li>Including this solo execution by A </li></ul>(1) (1) 0
  42. 43. Both Inputs 1 <ul><li>All executions must decide 1 </li></ul>1 1 (2) (1)
  43. 44. Both Inputs 1 <ul><li>Including this solo execution by B </li></ul>(1) (1) 1
  44. 45. What if inputs differ? (2) 1 0 By Way of contradiction ( 否定 ) : If univalent all executions must decide on same value
  45. 46. The Possible Executions <ul><li>Include the solo execution by A </li></ul><ul><li>that decides 0 </li></ul>0 1 (2)
  46. 47. The Possible Executions <ul><li>Also include the solo execution by B </li></ul><ul><li>which we know decides 1 </li></ul>0 1 (2)
  47. 48. Possible Executions Include <ul><li>Solo execution by A must decide 0 </li></ul><ul><li>Solo execution by B must decide 1 </li></ul>0 1 How univalent is that? (QED)
  48. 49. Critical States 次のステップで状態が絞り込まれていくステートのこと 0-valent 1-valent critical (3) (3)
  49. 50. From a Critical State A か B の決定が先勝ちで決まる状態 c If A goes first, protocol decides 0 If B goes first, protocol decides 1 0-valent 1-valent
  50. 51. Reaching Critical State 相手側が決定しないうちはまだ univalent これはだいたいの概念図 C A C A C B c C B univalent univalent univalent univalent 0-valent 1-valent initially bivalent
  51. 52. Critical States <ul><li>Starting from a bivalent initial state </li></ul><ul><li>The protocol can reach a critical state </li></ul><ul><ul><li>Otherwise we could stay bivalent forever </li></ul></ul><ul><ul><li>And the protocol is not wait-free </li></ul></ul>
  52. 53. Model Dependency <ul><li>So far, memory-independent! </li></ul><ul><li>True for </li></ul><ul><ul><li>Registers </li></ul></ul><ul><ul><li>Message-passing </li></ul></ul><ul><ul><li>Carrier pigeons </li></ul></ul><ul><ul><li>Any kind of asynchronous computation </li></ul></ul><ul><li>メモリそのものに特殊性は要求しないので、レジスタ、伝書鳩、メッセージング、全ての非同期に適応可能 </li></ul>
  53. 54. Read-Write Memory <ul><li>Reads and/or writes </li></ul><ul><li>To same/different registers </li></ul>
  54. 55. Completing the Proof <ul><li>Lets look at executions that: </li></ul><ul><ul><li>Start from a critical state </li></ul></ul><ul><ul><li>Threads cause state to become univalent by reading or writing to same/different registers </li></ul></ul><ul><ul><li>End within a finite number of steps deciding either 0 or 1 </li></ul></ul><ul><li>Show this leads to a contradiction </li></ul>
  55. 56. Possible Interactions ? ? ? ? y.write() ? ? ? ? x.write() ? ? ? ? y.read() ? ? ? ? x.read() y.write() x.write() y.read() x.read() A reads x A reads y
  56. 57. Some Thread Reads A runs solo, eventually decides 0 B reads x 1 0 A runs solo, eventually decides 1 c States look the same to A Contradiction
  57. 58. Possible Interactions ? ? no no y.write() ? ? no no x.write() no no no no y.read() no no no no x.read() y.write() x.write() y.read() x.read()
  58. 59. Writing Distinct Registers A writes y B writes x 1 0 c The song is the same A writes y B writes x Contradiction
  59. 60. Possible Interactions ? no no no y.write() no ? no no x.write() no no no no y.read() no no no no x.read() y.write() x.write() y.read() x.read()
  60. 61. Writing Same Registers States look the same to A A writes x B writes x 1 A runs solo, eventually decides 1 c 0 A runs solo, eventually decides 0 A writes x Contradiction
  61. 62. That’s All, Folks! QED no no no no y.write() no no no no x.write() no no no no y.read() no no no no x.read() y.write() x.write() y.read() x.read()
  62. 63. Recap: Atomic Registers Can’t Do Consensus  アトミックなレジスターのみではまだ足りない <ul><li>If protocol exists </li></ul><ul><ul><li>It has a bivalent initial state </li></ul></ul><ul><ul><li>Leading to a critical state </li></ul></ul><ul><li>What’s up with the critical state? </li></ul><ul><ul><li>Case analysis for each pair of methods </li></ul></ul><ul><ul><li>As we showed, all lead to a contradiction </li></ul></ul>
  63. 64. What Does Consensus have to do with Concurrent Objects?
  64. 65. Consensus Object public interface Consensus { Object decide(object value); } (4) (4)
  65. 66. Concurrent Consensus Object <ul><li>We consider only one time objects: each thread can execute a method only once </li></ul><ul><li>Linearizable to sequential consensus object in which </li></ul><ul><ul><li>the thread who’s input was decided on completed its method first </li></ul></ul>
  66. 67. Java Jargon Watch <ul><li>Define Consensus protocol as an abstract class </li></ul><ul><li>We implement some methods </li></ul><ul><li>Leave you to do the rest … </li></ul>
  67. 68. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4)
  68. 69. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4) Each thread’s proposed value
  69. 70. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4) Propose a value
  70. 71. Generic Consensus Protocol abstract class ConsensusProtocol implements Consensus { protected Object[] proposed = new Object[N]; private void propose(Object value) { proposed[ThreadID.get()] = value; } abstract public Object decide(object value); }} (4) (4) Decide a value: abstract method means subclass does the heavy lifting (real work)
  71. 72. Can a FIFO Queue Implement Consensus?
  72. 73. FIFO Consensus proposed array FIFO Queue with red and black balls Coveted red ball Dreaded black ball 8
  73. 74. Protocol: Write Value to Array 0 1 0
  74. 75. Protocol: Take Next Item from Queue 0 0 1 8
  75. 76. Protocol: Take Next Item from Queue 0 1 I got the coveted( 熱望 ) red ball, so I will decide my value I got the dreaded( 恐れる ) black ball, so I will decide the other’s value from the array <ul><li>ハズレを引くことで、相手が有効だとわかる </li></ul>8
  76. 77. Consensus Using FIFO Queue public class QueueConsensus extends ConsensusProtocol { private Queue queue; public QueueConsensus() { queue = new Queue(); queue.enq(Ball.RED); queue.enq(Ball.BLACK); } … }
  77. 78. Initialize Queue public class QueueConsensus extends ConsensusProtocol { private Queue queue; public QueueConsensus() { this .queue = new Queue(); this .queue.enq(Ball.RED); this .queue.enq(Ball.BLACK); } … } 8
  78. 79. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this .queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } }
  79. 80. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this .queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-ij]; } } Race to dequeue first queue item
  80. 81. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } } i = ThreadID.get(); I win if I was first
  81. 82. Who Won? public class QueueConsensus extends ConsensusProtocol { private Queue queue; … public decide(object value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } } Other thread wins if I was second
  82. 83. Why does this Work? <ul><li>If one thread gets the red ball </li></ul><ul><li>Then the other gets the black ball </li></ul><ul><li>Winner decides her own value </li></ul><ul><li>Loser can find winner’s value in array </li></ul><ul><ul><li>Because threads write array </li></ul></ul><ul><ul><li>Before dequeueing from queue </li></ul></ul>
  83. 84. Theorem <ul><li>We can solve 2-thread consensus using only </li></ul><ul><ul><li>A two-dequeuer queue, and </li></ul></ul><ul><ul><li>Some atomic registers </li></ul></ul><ul><li>Dequeuer がふたつあっても、自分が黒を引いたときは他がアカを引き当てたと想定できるので協調的に動作できる。 </li></ul>
  84. 85. Implications( 関わり合い ) <ul><li>Given </li></ul><ul><ul><li>A consensus protocol from queue and registers </li></ul></ul><ul><li>Assume there exists </li></ul><ul><ul><li>A queue implementation from atomic registers </li></ul></ul><ul><li>Substitution yields: </li></ul><ul><ul><li>A wait-free consensus protocol from atomic registers </li></ul></ul>contradiction (1) (1)
  85. 86. Corollary( 結論 ) <ul><li>It is impossible to implement </li></ul><ul><ul><li>a two-dequeuer wait-free FIFO queue </li></ul></ul><ul><ul><li>from read/write memory. </li></ul></ul><ul><li>Wait free の 2 つの dequeuer は整合性がとれないので、構成不可能。 </li></ul>
  86. 87. Consensus Numbers <ul><li>An object X has consensus number n </li></ul><ul><ul><li>If it can be used to solve n -thread consensus </li></ul></ul><ul><ul><ul><li>Take any number of instances of X </li></ul></ul></ul><ul><ul><ul><li>together with atomic read/write registers </li></ul></ul></ul><ul><ul><ul><li>and implement n -thread consensus </li></ul></ul></ul><ul><ul><li>But not (n+1) -thread consensus </li></ul></ul>
  87. 88. Consensus Numbers <ul><li>Theorem </li></ul><ul><ul><li>Atomic read/write registers have consensus number 1 </li></ul></ul><ul><li>Theorem </li></ul><ul><ul><li>Multi-dequeuer FIFO queues have consensus number at least 2 </li></ul></ul>
  88. 89. Consensus Numbers Measure Synchronization Power <ul><li>Theorem </li></ul><ul><ul><li>If you can implement X from Y </li></ul></ul><ul><ul><li>And X has consensus number c </li></ul></ul><ul><ul><li>Then Y has consensus number at least c </li></ul></ul>
  89. 90. Synchronization Speed Limit <ul><li>Conversely </li></ul><ul><ul><li>If X has consensus number c </li></ul></ul><ul><ul><li>And Y has consensus number d < c </li></ul></ul><ul><ul><li>Then there is no way to construct a wait-free implementation of X by Y </li></ul></ul><ul><li>This theorem will be very useful </li></ul><ul><ul><li>Unforeseen practical implications! </li></ul></ul><ul><li>コンセンサス数は減らすことは出来ない </li></ul>Theoretical Caveat: Certain weird exceptions exist 不整合時は論理的停止 する
  90. 91. Earlier Grand Challenge <ul><li>Snapshot means </li></ul><ul><ul><li>Write any array element </li></ul></ul><ul><ul><li>Read multiple array elements atomically </li></ul></ul><ul><li>What about </li></ul><ul><ul><li>Write multiple array elements atomically </li></ul></ul><ul><ul><li>Scan any array elements </li></ul></ul><ul><li>Call this problem multiple assignment </li></ul>
  91. 92. Multiple Assignment Theorem <ul><li>Atomic registers cannot implement multiple assignment </li></ul><ul><li>Weird or what? </li></ul><ul><ul><li>Single location write/multi location read OK </li></ul></ul><ul><ul><li>Multi location write/multi location read impossible </li></ul></ul><ul><li>一つなら可能だが、 Array の一括更新はアトミックには出来ない </li></ul>(1) (1)
  92. 93. Proof Strategy  不明 <ul><li>If we can write to 2/3 array elements </li></ul><ul><ul><li>We can solve 2-consensus </li></ul></ul><ul><ul><li>Impossible with atomic registers </li></ul></ul><ul><li>Therefore </li></ul><ul><ul><li>Cannot implement multiple assignment with atomic registers </li></ul></ul>(1)
  93. 94. Proof Strategy <ul><li>Take a 3 -element array </li></ul><ul><ul><li>A writes atomically to slots 0 and 1 </li></ul></ul><ul><ul><li>B writes atomically to slots 1 and 2 </li></ul></ul><ul><ul><li>Any thread can scan any set of locations </li></ul></ul>(1)
  94. 95. Double Assignment Interface interface Assign2 { public void assign( int i 1 , int v 1 , int i 2 , int v 2 ); public int read( int i); } (4) (4)
  95. 96. Double Assignment Interface interface Assign2 { public void assign( int i 1 , int v 1 , int i 2 , int v 2 ); public int read(int i); } (4) (4) Atomically assign value[i 1 ]= v 1 value[i 2 ]= v 2
  96. 97. Double Assignment Interface interface Assign2 { public void assign(int i 1 , int v 1 , int i 2 , int v 2 ); public int read( int i); } (4) (4) Return i-th value
  97. 98. Initially Writes to 0 and 1 Writes to 1 and 2 A B <ul><li>一つのスレッドが二つ以上を書く場合の競合の解説 </li></ul>
  98. 99. Thread A wins if A B (1) Thread B didn’t move
  99. 100. Thread A wins if A B (1) Thread B moved later
  100. 101. Thread A loses if A B (1) Thread B moved earlier 先に書いた方の勝ち
  101. 102. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; ←j が突然出てくる ? }} (4) (4)
  102. 103. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Extends ConsensusProtocol Decide sets j=i-1 and proposes value
  103. 104. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Three slots initialized to EMPTY
  104. 105. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Assign id 0 to entries 0,1 (or id 1 to entries 1,2)
  105. 106. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Read the register my thread didn’t assign
  106. 107. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY ||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Other thread didn’t move, so I win
  107. 108. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY|| other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Other thread moved later so I win
  108. 109. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) OK, I win.
  109. 110. Multi-Consensus Code class MultiConsensus extends …{ Assign2 a = new Assign2(3, EMPTY); public Object decide(object value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} (4) (4) Other thread moved first, so I lose (1)
  110. 111. Summary <ul><li>If a thread can assign atomically to 2 out of 3 array locations </li></ul><ul><li>Then we can solve 2 -consensus </li></ul><ul><li>Therefore </li></ul><ul><ul><li>No wait-free multi-assignment </li></ul></ul><ul><ul><li>From read/write registers </li></ul></ul><ul><li>従って 3 つの Array でもコンセンサスプロトコルは成立する </li></ul>
  111. 112. Read-Modify-Write Objects <ul><li>Method call </li></ul><ul><ul><li>Returns object’s prior value x </li></ul></ul><ul><ul><li>Replaces x with mumble(x) </li></ul></ul>
  112. 113. Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = this. value; this. value = mumble( this. value); return prior; } } (1)
  113. 114. Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = this. value; this.value = mumble(this.value); return prior; } } (1) Return prior value
  114. 115. Read-Modify-Write public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = this.value; this. value = mumble( this. value); return prior; } } (1) Apply function to current value
  115. 116. RMW Everywhere! <ul><li>Most synchronization instructions </li></ul><ul><ul><li>are RMW methods </li></ul></ul><ul><li>The rest </li></ul><ul><ul><li>Can be trivially transformed into RMW methods </li></ul></ul>
  116. 117. Example: Read public abstract class RMWRegister { private int value; public int synchronized read() { int prior = this. value; this. value = this. value; return prior; } } (1)
  117. 118. Example: Read public abstract class RMW { private int value; public int synchronized read() { int prior = this.value; this. value = this. value; return prior; } } (1) Apply f(v)=v , the identity function
  118. 119. Example: getAndSet public abstract class RMWRegister { private int value; public int synchronized getAndSet( int v) { int prior = this. value; this. value = v; return prior; } … } (1)
  119. 120. Example: getAndSet (swap) public abstract class RMWRegister { private int value; public int synchronized getAndSet(int v) { int prior = this.value; this. value = v; return prior; } … } (1) F(x)=v is constant function
  120. 121. getAndIncrement public abstract class RMWRegister { private int value; public int synchronized getAndIncrement() { int prior = this. value; this. value = this .value + 1; return prior; } … } (1)
  121. 122. getAndIncrement public abstract class RMWRegister { private int value; public int synchronized getAndIncrement() { int prior = this.value; this. value = this .value + 1; return prior; } … } (1) F(x) = x+1
  122. 123. getAndAdd public abstract class RMWRegister { private int value; public int synchronized getAndAdd( int a) { int prior = this. value; this. value = this .value + a; return prior; } … } (1)
  123. 124. Example: getAndAdd public abstract class RMWRegister { private int value; public int synchronized getAndIncrement(int a) { int prior = this.value; this. value = this .value + a; return prior; } … } (1) F(x) = x+a
  124. 125. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet( int expected, int update) { int prior = this. value; if ( this .value==expected) { this .value = update; return true ; } return false ; } … } (1)
  125. 126. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if ( this .value==expected ) { this.value = update; return true; } return false; } … } (1) If value is what was expected, …
  126. 127. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update ) { int prior = this.value; if (this.value==expected) { this .value = update; return true; } return false; } … } (1) … replace it
  127. 128. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if (this.value==expected) { this.value = update; return true ; } return false; } … } (1) Report success
  128. 129. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if (this.value==expected) { this.value = update; return true; } return false ; } … } (1) Otherwise report failure
  129. 130. Read-Modify-Write public abstract class RMWRegister { private int value; public void synchronized getAndMumble() { int prior = this.value; this.value = mumble( this. value); return prior; } } (1) Lets characterize F(x)…
  130. 131. Definition <ul><li>A RMW method </li></ul><ul><ul><li>With function mumble(x) </li></ul></ul><ul><ul><li>is non-trivial if there exists a value v </li></ul></ul><ul><ul><li>Such that v ≠ mumble(v) </li></ul></ul>
  131. 132. Par Example <ul><li>Identity(x) = x </li></ul><ul><ul><li>is trivial </li></ul></ul><ul><li>getAndIncrement(x) = x+1 </li></ul><ul><ul><li>is non-trivial </li></ul></ul>
  132. 133. Theorem <ul><li>Any non-trivial RMW object has consensus number at least 2 </li></ul><ul><li>No wait-free implementation of RMW registers from atomic registers </li></ul><ul><li>Hardware RMW instructions not just a convenience </li></ul><ul><li>ハードウェアによるアトミックレジスターもある、それにより利便性のみならず高速性が期待できる。 </li></ul><ul><li>RMW(Read-Modify-Write) は、この本による造語 </li></ul>
  133. 134. Reminder <ul><li>Subclasses of consensus have </li></ul><ul><ul><li>propose(x) method </li></ul></ul><ul><ul><ul><li>which just stores x into proposed[i] </li></ul></ul></ul><ul><ul><ul><li>built-in method </li></ul></ul></ul><ul><ul><li>decide(object value) method </li></ul></ul><ul><ul><ul><li>which determines winning value </li></ul></ul></ul><ul><ul><ul><li>customized, class-specific method </li></ul></ul></ul>
  134. 135. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4)
  135. 136. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) Initialized to v
  136. 137. Proof public class RMWConsensus extends Consensus { private RMWRegister r = v; public Object decide(object value) { if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) Am I first?
  137. 138. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) Yes, return my input
  138. 139. Proof public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public Object decide(object value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }} (4) No, return other’s input
  139. 140. Proof <ul><li>We have displayed </li></ul><ul><ul><li>A two-thread consensus protocol </li></ul></ul><ul><ul><li>Using any non-trivial RMW object </li></ul></ul><ul><li>レジスター値の置換はアトミックに実行可能な証明。 </li></ul>
  140. 141. Interfering RMW <ul><li>Let F be a set of functions such that for all f i and f j , either </li></ul><ul><ul><li>Commute: f i (f j (v))=f j (f i (v)) </li></ul></ul><ul><ul><li>Overwrite: f i (f j (v))=f i (v) </li></ul></ul><ul><li>Claim: Any set of RMW objects that commutes or overwrites has consensus number exactly 2 </li></ul>
  141. 142. Examples <ul><li>“ test-and-set” getAndSet(1) f(v)=1 </li></ul><ul><li>“ swap” getAndSet(x) f(v,x)=x </li></ul><ul><li>“ fetch-and-inc” getAndIncrement() f(v)=v+1 </li></ul><ul><ul><li>Overwrite f i (f j (v))=f i (v) </li></ul></ul><ul><ul><li>Overwrite f i (f j (v))=f i (v) </li></ul></ul><ul><ul><li>Commute f i (f j (v))= f j (f i (v)) </li></ul></ul><ul><li>コンセンサス数が 2 ならば、アトミックに出来ると言っている。 </li></ul>
  142. 143. Meanwhile Back at the Critical State c 0-valent 1-valent A about to apply f A B about to apply f B
  143. 144. Maybe the Functions Commute c 0-valent A applies f A B applies f B A applies f A B applies f B 0 1 C runs solo C runs solo 1-valent
  144. 145. Maybe the Functions Commute c 0-valent A applies f A B applies f B A applies f A B applies f B 0 1 C runs solo C runs solo 1-valent These states look the same to C Contradiction
  145. 146. Maybe the Functions Overwrite c 0-valent A applies f A B applies f B A applies f A 0 1 C runs solo C runs solo 1-valent
  146. 147. Maybe the Functions Overwrite c 0-valent A applies f A B applies f B A applies f A 0 1 C runs solo C runs solo 1-valent These states look the same to C Contradiction
  147. 148. Impact <ul><li>Many early machines provided these “weak” RMW instructions </li></ul><ul><ul><li>Test-and-set (IBM 360) </li></ul></ul><ul><ul><li>Fetch-and-add (NYU Ultracomputer) </li></ul></ul><ul><ul><li>Swap (Original SPARCs) </li></ul></ul><ul><li>We now understand their limitations </li></ul><ul><ul><li>But why do we want consensus anyway? </li></ul></ul>
  148. 149. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet( int expected, int update) { int prior = this. value; if ( this .value==expected) { this .value = update; return true ; } return false ; } … } (1)
  149. 150. compareAndSet public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if ( this .value==expected ) { this.value = update; return true; } return false; } … } (1) replace value if its what we expected, …
  150. 151. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } } (4)
  151. 152. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value) r.compareAndSet(-1,i); return proposed[r.get()]; } } (4) Initialized to -1
  152. 153. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } } (4) Try to swap in my id
  153. 154. compareAndSet Has ∞ Consensus Number public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public Object decide(object value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } } (4) Decide winner’s preference
  154. 155. The Consensus Hierarchy 1 Read/Write Registers, Snapshots… 2 getAndSet, getAndIncrement, … ∞ compareAndSet,… . . .
  155. 156. Multiple Assignment <ul><li>Atomic k -assignment </li></ul><ul><li>Solves consensus for 2k-2 threads </li></ul><ul><li>Every even consensus number has an object (can be extended to odd numbers) </li></ul>
  156. 157. Lock-Freedom <ul><li>Lock-free: in an infinite execution infinitely often some method call finishes (obviously, in a finite number of steps) </li></ul><ul><li>Pragmatic approach </li></ul><ul><li>Implies no mutual exclusion </li></ul>
  157. 158. Lock-free と Wait-free <ul><li>Lock-free と Wait-free アルゴリズム とは、共有データにロックをかけて保護するアルゴリズムとは違い、複数のスレッドが同 時並行的に、ある対象データを壊すことなしに読み書きすることを可能にするアルゴリズムである。 Lock-free とはスレッドがロックしないことを意味しており、全てのステップにおいてシステムが必ず進行する。通常ロックを所有するスレッドは、コンテキストスイッチ が発生した時点で、全体の進行を阻止することがあるので、 Look-free ではミューテックスやセマフォといった、排他制御のためのプリミティブが使われていないということになる。 Wait-free とは、他のスレッドの動作に関係なく、スレッドがいかなる操作も有限のステップで操作を完了させられることを指す。全てのアルゴリズムを Lock-free にすることが出来るが、一部しか Wait-free にできない。 Wait-free なアルゴリズムは Lock-free である。 </li></ul>
  158. 159. Lock-free への書き換え <ul><li>例えば、銀行口座への預金プログラムを作るとする。それぞれのスレッドを ATM とする。預金預け入れをするには、現在の預金残高を読み出し、預入金額を足し算し、新しい預金残高を書き込むという処理が必要である。ロック方式のやり方の場合、 1 つ目の ATM が預金をするために、口座にロックをかけ、 2 つ目の ATM が同時に預金残高を変更できないようにロックをかける。さもないと、同時に処理してしまうと、最終的な預金残高に不整合が起きうる。この処理を Lock-free にするには、すべての預入要求を管理するスレッドを作り、そこに、 Wait-free のキューを作り、 ATM はそのキューに対して非同期にロックをかけることなく預入要求を入れ、預入要求を管理するスレッドはキューから順次取り出し、預金残高を更新する。このやり方の方が、わざわざ Lock-free の預金アルゴリズムを作るよりも、プログラミングは楽である。さらに、この手法は、キューが Wait-free であるので、 Lock-free なだけでなく、 Wait-free でもある。預金残高の書き換え処理を n 並列で行いたいなら、 n 個 Wait-free キューを作り、口座番号を n で割った余りでどのキューに入れるか決めるという方法で対応できる。 </li></ul>
  159. 160. Lock-free な口座入金 <ul><li>Wait-free な queue が ATM 台数分ある。 </li></ul><ul><li>入金処理はそれぞれの queue が wait-free で受け付ける。 </li></ul><ul><li>対象のひとつの口座の更新は、 Lock-free に cas で更新をかける、 cas 更新は待たせないが、失敗することもある。 </li></ul><ul><li>したがって wait-free ではない、失敗したら再試行する必要があるためだ。 </li></ul><ul><li>しかし synchronozed メソッドよりは格段にスケールする。 </li></ul>
  160. 161. Lock-Free vs. Wait-free <ul><li>Wait-Free: each method call takes a finite number of steps to finish </li></ul><ul><ul><li>Lock を使って、資源を排他管理する </li></ul></ul><ul><li>Lock-free: infinitely often some method call finishes </li></ul><ul><ul><li>Lock なしは、常に成立するわけではない </li></ul></ul><ul><ul><li>compareAndSwap() でロックなし更新する </li></ul></ul>
  161. 162. Lock-Freedom <ul><li>Any wait-free implementation is lock-free . </li></ul><ul><li>Lock-free is the same as wait-free if the execution is finite. </li></ul><ul><li>Old saying: “ Lock-free is to wait-free as deadlock-free is to lockout-free .” </li></ul>
  162. 163. Lock-Freedom <ul><li>wait-free ならそれは lock-free . </li></ul><ul><li>Lock-free は wait-free と等価 . </li></ul><ul><li>すべてを Lock-free にできるわけではない . </li></ul><ul><li>Old saying: “ Lock-free is to wait-free as deadlock-free is to lockout-free .” </li></ul>
  163. 164. Lock-Free Implementations <ul><li>Lock-free consensus is as impossible as wait-free consensus </li></ul><ul><li>All the results we presented hold for lock-free algorithms also. </li></ul>(2)
  164. 165. There is More: Universality <ul><li>Consensus is universal </li></ul><ul><li>From n -thread consensus </li></ul><ul><ul><li>Wait-free/Lock-free </li></ul></ul><ul><ul><li>Linearizable </li></ul></ul><ul><ul><li>n -threaded </li></ul></ul><ul><ul><li>Implementation </li></ul></ul><ul><ul><li>Of any sequentially specified object </li></ul></ul>Stay tuned…
  165. 166. Java.util.concurrent を使おう <ul><li>++ の操作は atomic ではないので危険 </li></ul>int counter = 0; … . counter++; import java.util.concurrent.atomic.AtomicInteger; AtomicInteger counter = new AtomicInteger(0); … . counter.getAndIncrement(); <ul><li>これなら複数スレッドから lock-free に更新可能 </li></ul>
  166. 167. Java.util.concurrent による並行制御 Thread-1 Thread- 2 Thread- 3 Abstract Synchronized Queue Abstract Synchronized Queue Abstract Synchronized Queue 更新先 先に完成したものを準備しておいて 一瞬で cas で切り替えることで Lock をかけない 各スレッドからの要求を Queue で受けて、待たせない
  167. 168.            This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License . <ul><li>You are free: </li></ul><ul><ul><li>to Share — to copy, distribute and transmit the work </li></ul></ul><ul><ul><li>to Remix — to adapt the work </li></ul></ul><ul><li>Under the following conditions: </li></ul><ul><ul><li>Attribution . You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). </li></ul></ul><ul><ul><li>Share Alike . If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. </li></ul></ul><ul><li>For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to </li></ul><ul><ul><li>http://creativecommons.org/licenses/by-sa/3.0/. </li></ul></ul><ul><li>Any of the above conditions can be waived if you get permission from the copyright holder. </li></ul><ul><li>Nothing in this license impairs or restricts the author's moral rights. </li></ul>

×