Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20141223 kuznetsov distributed

804 views

Published on

Kuznetsov
Dsitributed

Published in: Science
  • Be the first to comment

  • Be the first to like this

20141223 kuznetsov distributed

  1. 1. 
 Mathematics in Synchronization and Concurrency 
 
 " Petr Kuznetsov" Telecom ParisTech" " " Algorithmic methods laboratory, PDMI, SPb, 2014" "
  2. 2. Télécom ParisTech" §  A.k.a. ENST (École nationale supérieure des télécommunications)" §  Member of ParisTech and Mines-Telecom "" §  One the most known “grandes écoles”" §  One of the founders of Eurécom" §  Research labs:" ü INFRES – computer science and networking" ü COMELEC – electronics and communication" ü TSI – signal and image processing " ü SES – economics and social sciences" §  Research internship/PhD at INFRES: petr.kuznetsov@telecom-paristech.fr" ©  2014  P.  Kuznetsov    
  3. 3. " " What is computing?" "
  4. 4. " " What is done by a Turing machine" " " " "
  5. 5. " Not well adjusted to concurrency?" " Computation as interaction" " Robin Milner" 1934-2010"
  6. 6. " " Distributed computing model:" independent sequential processes that communicate" " " ©  2014  P.  Kuznetsov    
  7. 7. Concurrency is everywhere!" §  Sensor networks" §  Internet" §  Basically everything related computing" •  Multi-core processors" ©  2014  P.  Kuznetsov    
  8. 8. Moore’s Law and CPU speed" ©  2014  P.  Kuznetsov    
  9. 9. 
 “washing machine science”?" §  Single-processor performance does not improve" §  But we can add more cores" §  Run concurrent code on multiple processors" Can we expect a proportional speedup? (ratio between sequential time and parallel time for executing a job)" ©  2014  P.  Kuznetsov    
  10. 10. Example: painting in parallel" §  3 friends want to paint 3 equal-size walls, one friend per wall" ü Speedup = 3" " §  What if one wall is twice as big?" ©  2014  P.  Kuznetsov    
  11. 11. Amdahl’s Law" §  p – fraction of the work that can be done in parallel (no synchronization)" §  n - the number of processors" §  Time one processor needs to complete the job = 1" ©  2014  P.  Kuznetsov    
  12. 12. A better solution" §  When done, help the others " ü All 3 paint the remaining half-room in parallel " §  Communication and agreement is required!" §  This is a hard task! §  Synchronization algorithms!" " ©  2014  P.  Kuznetsov    
  13. 13. Why synchronize ?" §  Concurrent accesses to a shared resource (e.g., a file) may lead to an inconsistent state " ü Non-deterministic result (race condition): the resulting state depends on the scheduling" §  Concurrent accesses need to be synchronized! ü E. g., decide who is allowed to update a given part of the file at a given time" §  Code leading to a race condition is called critical section! ü Must be (or appear to be) executed sequentially" §  Synchronization problems: mutual exclusion, readers- writers, producer-consumer, …" §  Synchronization abstractions: semaphores, TAS, CAS, queues, skip-lists, dictionaries, transactional memory " ©  2014  P.  Kuznetsov    
  14. 14. Challenges" §  What is correct?" ü Safety and liveness" §  What is the cost of synchronization?" ü Time and space lower bounds" §  Failures/asynchrony" ü Fault-tolerant synchronization?" §  How to distinguish possible from impossible? " ü Impossibility results" " ©  2014  P.  Kuznetsov    
  15. 15. Distributed ≠ Parallel" §  The main challenge is synchronization despite uncertainty (failures, asynchrony)" " §  “you know you have a distributed system when the crash of a computer you’ve never heard of stops you from getting any work done” (Lamport)" ©  2014  P.  Kuznetsov    
  16. 16. Why theory of distributed computing?" §  Every computing system is distributed" §  Computing getting mission-critical" ü Understanding fundamentals is crucial" §  Intellectual challenge " ü A distinct math domain?" ©  2014  P.  Kuznetsov    
  17. 17. History" §  Dining philosophers, mutual exclusion (Dijkstra )~60’s" §  Distributed computing, logical clocks (Lamport), distributed transactions (Gray) ~70’s" §  Consensus (Lynch) ~80’s" §  Distributed programming models ~90’s" §  Multicores, HTM now" §  Principal conferences: PODC, DISC, SPAA, ICDCS, STOC, FOCS, …" ©  2014  P.  Kuznetsov    
  18. 18. Synchronization, blocking 
 and non-blocking
 "
  19. 19. Producer-consumer (bounded buffer) problem §  Producers put items in the buffer (of bounded size)" §  Consumers get items from the buffer" §  Every item is consumed, no item is consumed twice" "(Client-server, multi-threaded web servers, pipes, …)" Why synchronization? Items can get lost or consumed twice:" " Producer! /* produce item */! while (counter==MAX);! buffer[in] = item ! in = (in+1) % MAX;! counter++; !! Consumer! /* to consume item */! while (counter==0); ! item=buffer[out];! out=(out+1) % MAX;! counter--; ! /* consume item */! ! Race! ©  2014  P.  Kuznetsov    
  20. 20. Synchronization tools §  Semaphores (locks), monitors §  Nonblocking synchronization §  Transactional memory ©  2014  P.  Kuznetsov    
  21. 21. Semaphores for producer-consumer §  2 semaphores used :" ü empty: indicates empty slots in the buffer (to be used by the producer)" ü full: indicates full slots in the buffer (to be read by the consumer)" shared semaphores empty := MAX, full := 0;! Producer" Consumer" P(empty)! buffer[in] = item; ! in = (in+1) % MAX;! V(full)! ! ! P(full);! item = buffer[out];! out=(out+1) % MAX; ! V(empty);! ©  2014  P.  Kuznetsov    
  22. 22. Potential problems with semaphores/locks §  Blocking: progress of a process is conditional (depends on other processes)" §  Deadlock: no progress ever made! " ! §  Starvation: waiting in the waiting queue forever" X1:=1; X2:=1! Process 1! Process 2! ...! P(X1)! P(X2)! critical section! V(X2)! V(X1)! ...! ...! P(X2)! P(X1)! critical section! V(X1)! V(X2)! ...! ©  2014  P.  Kuznetsov    
  23. 23. Non-blocking algorithms" A process makes progress, regardless of the other processes" shared buffer[MAX]:=empty; head:=0; tail:=0;" Producer put(item)! Consumer get()! if (tail-head == MAX){! !return(full);! }! buffer[tail%MAX]=item; ! tail++;! return(ok);! if (tail-head == 0){! !return(empty);! }! item=buffer[head%MAX]; ! head++;! return(item);! Problems: " •  works for 2 processes but hard to say why it works J" •  multiple producers/consumers? Other synchronization pbs?" (highly non-trivial in general)" ©  2014  P.  Kuznetsov    
  24. 24. Transactional memory" §  Mark sequences of instructions as an atomic transaction, e.g., the resulting producer code:" atomic {! !if (tail-head == MAX){! !return full;! !}! !items[tail%MAX]=item; ! !tail++;! }! return ok;! §  A transaction can be either committed or aborted! ü Committed transactions are serializable ! ü + aborted/incomplete transactions are consistent (opacity/local serializability)! ü Let the transactional memory (TM) care about the conflicts" ü Easy to program, but performance may be problematic" ©  2014  P.  Kuznetsov    
  25. 25. §  Concurrency is indispensable in programming:" ü Every system is now concurrent" ü Every parallel program needs to synchronize" ü Synchronization cost is high (“Amdahl’s Law”)" §  Tools: " ü Synchronization primitives (e.g., monitors, TAS, CAS, LL/SC)" ü Synchronization libraries (e.g., java.util.concurrent)" ü Transactional memory, also in hardware (Intel Haswell, IBM Blue Gene,…)" §  Coming next:" ü  Characterization of implementability in shared-memory models" " ©  2014  P.  Kuznetsov    
  26. 26. Characterization of Distributed Computing Models
 "
  27. 27. t-­‐resilience   CAS   Locks   Transac>onal   memory   Message-­‐ passing    Shared-­‐ memory   Which features matter?" Matter for what?" Synchronization jungle"  Adversaries  
  28. 28. A characterization?" " §  finding an equivalent (simpler) representation of an object" " §  creating characters for a narrative" ü Which characters matter? Equivalence results?" ©  2014  P.  Kuznetsov    
  29. 29. A distributed computability play" Characters: problems and models" Intrigue: is P solvable in M? " " How many interesting plays are there?" "" §  Problems: objects (queues, stacks, CAS), distributed tasks" §  Models: message-passing, shared-memory, wait-freedom, t- resilience, adversaries" J.L.Borges:" there are only four stories to tell" ©  2014  P.  Kuznetsov    
  30. 30. Task solvability in read-write" §  A task (I,O,Δ):" ü Set of input vectors I" ü Set of output vectors O" ü Task specification Δ: I è 2O" §  Consensus (universal task): " ü Binary inputs and outputs" ü Every output is an input (validity)" ü All outputs are the same (agreement)" " " ©  2014  P.  Kuznetsov     Before   AHer   0 1 2 2 2 2
  31. 31. System model" §  N asynchronous (no bounds on relative speeds) processes p0,…,pN-1 (N≥2) communicate via read- write shared memory" §  Every process pi is assigned an algorithm Ai" §  Processes can fail by crashing " ü A crashed process takes only finitely many steps (reads and writes)" ü Up to t processes can crash: t-resilient system " ü t=N-1: wait-free " " ©  2014  P.  Kuznetsov    
  32. 32. Solving a task" " §  A task T is solvable if for some distributed algorithm {A0,…,AN-1} and every input vector I in I:" ü Every correct process eventually outputs a value (decides)" ü The output vector O is in Δ(I) " ©  2014  P.  Kuznetsov    
  33. 33. An easy story to tell" " Task solvability in the iterated immediate snapshot (IIS) memory model " §  Processes proceed in rounds" §  Each round a new memory is accessed:" ü Write the current state" ü Get a snapshot of the memory " ü Equivalent the standard (non-iterated) read-write model [BGK14]" §  Visualized via standard chromatic subdivision χk(I) [BG97,Lin09,Koz14]" ©  2014  P.  Kuznetsov    
  34. 34. Iterated standard chromatic subdivision: χ0(I)" p1   p3   p2  
  35. 35. χ1(I) : one round of IIS" p1   p3   p2   p2  sees  {p1,p2}   p1  sees  {p1,p2}   p3  sees  {p1,p2,p3}   p2  sees  {p2,p3}   p3  sees  {p2,p3}  
  36. 36. χ2(I) : two rounds of IIS" p1   p3   p2  
  37. 37. Impossibility of wait-free consensus [FLP85,LA87]" Theorem 1 No wait-free algorithm solves consensus" " We give the proof for N=2, assuming that " "p0 proposes 0 and p1 proposes 1" " " Implies the claim for all N≥2" " ©  2014  P.  Kuznetsov    
  38. 38. Proof of Theorem 1" p0   p1   p0  reads  before     p1  writes   p0  reads  a0er     p1  writes   p1  reads  a0er     p0  writes   p1  reads  before     p0  writes   Initially each pi only knows its input" One round of IIS:"
  39. 39. Proof of Theorem 1" p0   p1   Two rounds of IIS:"
  40. 40. Proof of Theorem 1" p0   p1   And so on…" Solo  runs  remain  connected  -­‐  no  way   to  decide!    
  41. 41. Proof of Theorem 1" p0   p1   Suppose pi (i=0,1) proposes i" §  pi must decide i in a solo run!" Suppose by round r every process decides" " There  exists  a  run  with  conflic>ng   decisions!   0   0   0   0   0   1  1  0   0   0   0   0   1  1  1  1  1  1  1  1  1  1  1  1  
  42. 42. Thus" §  Wait-free consensus is impossible (for all N)" §  What about weaker forms of agreement?" ü E.g., decide on at most k (k<N) different values: k-set consensus" ü k=N-1: set consensus" " Before   AHer   0 1 2 1 2 2
  43. 43. Impossibility of set consensus" Theorem 2 No wait-free algorithm solves set agreement in read-write" " §  Starting with N values, there is no way to drop one (decide on at most N-1)" " §  Implies the impossibility of wait-free k-set consensus for all k≤N-1"
  44. 44. Solving set consensus in IIS" §  Each pi proposes its id i" §  There is an IIS round r, such that each process (that reaches round r) outputs some id " §  In each run, the set of output ids is a subset of participants of size ≤N-1" " Implies Sperner coloring of the subdivided simplex IISr "
  45. 45. Sperner’s coloring of a subdivided simplex" Each vertex of the original simplex SN-1 (initial state) is colored with a process id " " Each simplex in subdivision D is contained in a face (subset) of S and the union of simplexes of D is S (|S|=|D|) " " Sperner’s coloring:" "Each vertex v in the subdivision D is colored with a distinct color:" ü If the vertex belongs to a face S of SN-1, then d is in colors(S) " " P0   P1   P2   P0   P1   P2   SN-­‐1   D  
  46. 46. Sperner’s Lemma" Sperner coloring of any subdivision D of SN-1 has a simplex with all N colors" " " " " Corollary: wait-free set consensus is impossible in IIS (and, thus, in read-write)" P0   P1   P2  
  47. 47. Characterization of wait-free task solvability" Asynchronous computability theorem[HS93]" A task (I,O,Δ) is wait-free RW solvable if and only if there is a chromatic simplicial map from a subdivision χk(I) to O carried by Δ" " Wait-free task computability reduced to the existence of continuous maps between geometrical objects" " "
  48. 48. 1-resilient consensus?
 k-resilient k-set agreement?"" " NO" " Operational model equivalence:" " §  t-resilient N-process ó wait-free (t+1)-process [BG93] (for colorless task, e.g. k-set agreement)" " [Main tool – protocol simulations]" "
  49. 49. What about…" §  Generic sub-models of RW" ü Many problems cannot be solved wait-free" ü So restrictions (sub-models) of RW were considered" ü E.g., adversarial models [DFGT09]: specifying the possible correct sets" ●  Non-uniform/correlated fault probabilities" ●  Hitting set of size 1: a leader can be elected" " "
  50. 50. Beyond wait-free and colorless tasks " §  A generic sub-IIS model is not compact: does not contain limit points " §  A protocol cannot be a approximated by a finite subdivision: colors do matter " ©  2014  P.  Kuznetsov    
  51. 51. Terminating subdivisions [GKM14]" A (possibly infinite) subdivision that allows processes to stabilize (models outputs): stable simplices stop subdividing" A terminating subdivision must be compatible with the model: every run eventually lands in a stable simplex " ©  2014  P.  Kuznetsov    
  52. 52. Generalized asynchronous computability theorem [GMK14]" A task (I,O,Δ) is solvable in a sub-IIS model M if and only if there exists a terminating subdivision T of I (compatible with M) and a chromatic simplicial map from stable simplices of T to O carried by Δ " " Topological characterization of every task facing every (crash) adversary?" What about stronger ones? What about long-lived problems?" " " ©  2014  P.  Kuznetsov    
  53. 53. Open problems" §  Modeling and characterization" ü Long-lived abstractions (queues, hash tables, TMs…)" ü Byzantine adversary: a faulty process deviates arbitrarily" ü Partial synchrony" §  Complexity bounds" ü Time (number of steps) and space (amount of memory/communication)" ü Optimal algorithm design" §  Mathematics induced by DC" ü filling the gap between the (too abstract) algebraic/combinatorial topology and algorithms" " ©  2014  P.  Kuznetsov    
  54. 54. ANR-DFG Cooperation 2014 
 
 Mathematical Methods in Distributed Computing (DISCMAT)" Partners:" " Petr Kuznetsov, Telecom ParisTech (coord.)" Dmitry Kozlov, U Bremen" Michel Raynal, U Rennes I" " " "
  55. 55. References" §  E. Gafni, P. Kuznetsov, C. Manolescu: A generalized asynchronous computability theorem. PODC 2014" §  Z. Bouzid, E. Gafni, P. Kuznetsov: Strong Equivalence Relations for Iterated Models. OPODIS 2014" §  M. Herlihy, S. Kozlov, S. Rajsbaum: Distributed computing through combinatorial topology. Morgan Kaufmann, 2014 " §  More: http://perso.telecom-paristech.fr/ ~kuznetso/ " ©  2014  P.  Kuznetsov    
  56. 56. " " " Спасибо!" "

×