Guided by                Presented by
Shri. R.S. Sharma     Ashutosh Jaiswal
Associate Prof, CSE     M.tech 2nd Yr
What is a snapshot
              
 A snapshot of a distributed system is a global state
  where the local states of all processes and of all
  communication channels are recorded
  simultaneously
Where is a snapshot
           used?
             
 Detection of deadlock of a distributed system
 Compute monotonic functions of the global state
  such as lower bounds on the simulation time.
 Check pointing and recovery of distributed data
  bases
 Monitoring and debugging of distributed systems.
Determining Global
           States
             
 Global State

  “The global state of a distributed system is the set of local
  states of all individual processes involved in the
  computation plus the state of the communication
  channels.”
Why global state determination is
 difficult in Distributed Systems?
                      
 Distributed State : Have to collect information
  that is spread across several machines!!
 Only Local knowledge : A process in the
  computation does not know the state of other
  processes.
 The lack of globally shared memory, global clock
  and unpredictable message delays in a
  distributed system make this problem non-
  trivial.
Consistent and
              Inconsistent cuts
                               arrow starts in
     A cut is consistent if no message
      future and ends in past. (e.g. ) AB
     Otherwise it is inconsistent ( e.g.) CD     C
                    A
1

2

3
4

     B                                 D
Global States of Consistent Cuts
                          
 A global state computed along a consistent cut is
  correct
 The global state of a consistent cut comprises the local
  state of each process at the time the cut event
  happens and the set of all messages sent but not yet
  received
 The snapshot problem consists in designing an efficient
  protocol which yields only consistent cuts and to
  collect the local state information
Issues in recording a global state
                          
 How to distinguish between the messages to be
  recorded in the snapshot from those not to be
  recorded.
   -Any message that is sent by a process before
    recording its snapshot, must be recorded in the global
    snapshot
   -Any message that is sent by a process after recording
    its snapshot , must not be recorded in the global
    snapshot
How to determine the instant
when a process takes its snapshot
                        
 A process pj must record its snapshot before
  processing a message mij that was sent by process pi
  after recording its snapshot.
Models of
          communication
              
There are two models of communication : FIFO, non-
FIFO
 In FIFO model, each channel acts as a first-in first-
   out message queue and thus, message ordering is
   preserved by a channel.
 In non-FIFO model, a channel acts like a set in which
   the sender process adds messages and the receiver
   process removes messages from it in a random order.
System Model for Global Snapshots

                               
 The system consists of a collection of n processes
  p1, p2, ..., pn that are connected by channels.
 There are no globally shared memory and physical
  global clock and processes communicate by passing
  messages through communication channels.
 Cij denotes the channel from process pi to process pj
  and its state is denoted by SCij .
 The actions performed by a process are modeled as
  three types of events:
    Internal events,the message send event and the message
     receive event.
    For a message mij that is sent by process pi to process pj , let
     send(mij ) and rec(mij ) denote its send and receive events.
Process States and Messages in transit

                          
 At any instant, the state of process pi , denoted by
  LSi , is a result of the sequence of all the events
  executed by pi till that instant.
 For an event e and a process state LSi , e∈LSi iff e
  belongs to the sequence of events that have taken
  process pi to state LSi .
 For an event e and a process state LSi , e (not in) LSi
  iff e does not belong to the sequence of events that
  have taken process pi to state LSi .
Snapshot algorithms
     for FIFO channels
             
                   Chandy-Lamport algorithm
   The Chandy-Lamport algorithm uses a control
    message, called a marker whose role in a FIFO system is
    to separate messages in the channels.
   After a site has recorded its snapshot, it sends a
    marker, along all of its outgoing channels before sending
    out any more messages.
   A marker separates the messages in the channel into those
    to be included in the snapshot from those not to be
    recorded in the snapshot.
   A process must record its snapshot no later than when it
    receives a marker on any of its incoming channels.

Snapshot algorithms for non-
       FIFO channels
                         
 In a non-FIFO system, a marker cannot be used to
  delineate messages into those to be recorded in the
  global state from those not to be recorded in the
  global state.
Lai-Yang algorithm
               
 The Lai-Yang algorithm fulfills this role of a marker in a non-
  FIFO system by using a coloring scheme on computation
  messages that works as follows:
   Every process is initially white and turns red while taking a
     snapshot. The equivalent of the “Marker Sending Rule” is
     executed when a process turns red.
   Every message sent by a white (red) process is colored
     white (red) indicating if it was sent before(after) snapshot.
   Each process (which is initially white) becomes red as soon
     as it receives a red message for the first time and starts a
     virtual broadcast algorithm to ensure that all processes will
     eventually become red.
Determining Messages
       in transit
           
 White process records history of white msgs
   sent/received on each channel.
 When a process turns red, it sends these histories along
   with its snapshot to the initiator process that collects the
   global snapshot.
 Initiator process evaluates transit(LSi , LSj ) to compute
   state of a channel Cij :
 SCij = white messages sent by pi on Cij − white messages
   received by pj on Cij =
 {send(mij )|send(mij ) ∈ LSi } − {rec(mij )|rec(mij ) ∈ LSj }.
The Proposed Algorithm
          
                      Assumptions:
 Author assumes a pre-established spanning tree on the
  distributed system for an initiator to perform a one-to-all
  broadcast to inform all other processes the initiation of a
  new global snapshot.
 In doing so, it can be ensured that even without receiving
  any red message, a white process will learn that one
  snapshot execution has been initiated and then will
  execute the global-snapshot algorithm to construct the
  global snapshot cooperatively with all other processes

 There are three phases in the algorithm
 Phase 1: This is the phase of snapshot initiation. The
  initiator performs a one-to-all broadcast of INIT
  control messages along the pre-established spanning
  tree to inform all other processes .

 Phase 2: This is the phase of accumulating numbers of
  white messages. Upon the receipt of an INIT control
  message or a red computation message, a white process
  turns red and records its local state.
 Subsequently, the algorithm directs the whole processes
  to calculate in a symmetrical manner the total number of
  white messages that each process is supposed to receive.
 In particular, every process pi maintains a vector of size
  N, wmsg senti[], to count in wmsg senti[j] the number of
  white messages that it has sent to another process pj .



 Phase 3: The is the phase of recording channel states.
  When a red process pi receives a white computation
  message along a channel, such a message is added to
  the state of the channel. In addition, when the second
  phase is done and all white messages supposed to be
  received by pi have been received by pi, i.e. wmsg
  receivedi = sum wmsgi[i], pi turns white and then
  terminates the algorithm locally.
Algorithm
                Initiator - 1
     3




2            4




         1
                 Initiator - 1
    3




2                    4




        1   Control Messages & a
            Red message sent
       Initiator - 1
                    3




                2               4



Red Message
Received at 2           1
                Initiator - 1
3 sends white   3   Ctrl Mesg from
  Mesg to 1           1 reaches 3



2                                4




                1
                        Initiator - 1
    3




2                       4

                            Red Ctrl
                              Mesg
    1                       received
        White Mesg
        received at 1
Summary
                   
 Global State detection difficult in Distributed
  Systems
 Snapshot algorithm may not give an actual state but
  is very helpful in detecting Stable Properties
References
                         
[1] Jichiang Tsai, “Flexible Symmetrical Global-Snapshot Algorithms for
Large-Scale Distributed Systems,” IEEE Transactions On Parallel And
Distributed Systems, vol. Xx, no. Y, apr. 2012
[2] R. Garg, V. Garg, and Y. Sabharwal, “Efficient algorithms for global
snapshots in large distributed systems,” IEEE Trans. Parallel and Distributed
Systems, vol. 21, no. 5, pp. 620–630, May 2010.
[3] A. D. Kshemkalyani, “Fast and message-efficient global snapshot
algorithms for large-scale distributed systems,” IEEE Trans. Parallel and
Distributed Systems, vol. 21, no. 9, pp. 1281–1209, Sept. 2010.
[4]F. Mattern, “Efficient Algorithms for Distributed Snapshots and Global
Virtual Time Approximation,” J. Parallel and Distributed Computing, pp.
423-434, Aug. 1993.
[5] K.M. Chandy and L. Lamport, “Distributed Snapshots: Determining
Global States of Distributed Systems,” ACM Trans. Computer Systems,
vol. 3, no. 1, pp. 63-75, Feb. 1985.

Thank You !

Flexible Symmetric Global Snapshot

  • 1.
    Guided by Presented by Shri. R.S. Sharma Ashutosh Jaiswal Associate Prof, CSE M.tech 2nd Yr
  • 2.
    What is asnapshot   A snapshot of a distributed system is a global state where the local states of all processes and of all communication channels are recorded simultaneously
  • 3.
    Where is asnapshot used?   Detection of deadlock of a distributed system  Compute monotonic functions of the global state such as lower bounds on the simulation time.  Check pointing and recovery of distributed data bases  Monitoring and debugging of distributed systems.
  • 4.
    Determining Global States   Global State “The global state of a distributed system is the set of local states of all individual processes involved in the computation plus the state of the communication channels.”
  • 5.
    Why global statedetermination is difficult in Distributed Systems?   Distributed State : Have to collect information that is spread across several machines!!  Only Local knowledge : A process in the computation does not know the state of other processes.  The lack of globally shared memory, global clock and unpredictable message delays in a distributed system make this problem non- trivial.
  • 6.
    Consistent and Inconsistent cuts  arrow starts in  A cut is consistent if no message future and ends in past. (e.g. ) AB  Otherwise it is inconsistent ( e.g.) CD C A 1 2 3 4 B D
  • 7.
    Global States ofConsistent Cuts   A global state computed along a consistent cut is correct  The global state of a consistent cut comprises the local state of each process at the time the cut event happens and the set of all messages sent but not yet received  The snapshot problem consists in designing an efficient protocol which yields only consistent cuts and to collect the local state information
  • 8.
    Issues in recordinga global state   How to distinguish between the messages to be recorded in the snapshot from those not to be recorded.  -Any message that is sent by a process before recording its snapshot, must be recorded in the global snapshot  -Any message that is sent by a process after recording its snapshot , must not be recorded in the global snapshot
  • 9.
    How to determinethe instant when a process takes its snapshot   A process pj must record its snapshot before processing a message mij that was sent by process pi after recording its snapshot.
  • 10.
    Models of communication  There are two models of communication : FIFO, non- FIFO  In FIFO model, each channel acts as a first-in first- out message queue and thus, message ordering is preserved by a channel.  In non-FIFO model, a channel acts like a set in which the sender process adds messages and the receiver process removes messages from it in a random order.
  • 11.
    System Model forGlobal Snapshots   The system consists of a collection of n processes p1, p2, ..., pn that are connected by channels.  There are no globally shared memory and physical global clock and processes communicate by passing messages through communication channels.  Cij denotes the channel from process pi to process pj and its state is denoted by SCij .  The actions performed by a process are modeled as three types of events:  Internal events,the message send event and the message receive event.  For a message mij that is sent by process pi to process pj , let send(mij ) and rec(mij ) denote its send and receive events.
  • 12.
    Process States andMessages in transit   At any instant, the state of process pi , denoted by LSi , is a result of the sequence of all the events executed by pi till that instant.  For an event e and a process state LSi , e∈LSi iff e belongs to the sequence of events that have taken process pi to state LSi .  For an event e and a process state LSi , e (not in) LSi iff e does not belong to the sequence of events that have taken process pi to state LSi .
  • 13.
    Snapshot algorithms for FIFO channels  Chandy-Lamport algorithm  The Chandy-Lamport algorithm uses a control message, called a marker whose role in a FIFO system is to separate messages in the channels.  After a site has recorded its snapshot, it sends a marker, along all of its outgoing channels before sending out any more messages.  A marker separates the messages in the channel into those to be included in the snapshot from those not to be recorded in the snapshot.  A process must record its snapshot no later than when it receives a marker on any of its incoming channels.
  • 14.
  • 15.
    Snapshot algorithms fornon- FIFO channels   In a non-FIFO system, a marker cannot be used to delineate messages into those to be recorded in the global state from those not to be recorded in the global state.
  • 16.
    Lai-Yang algorithm   The Lai-Yang algorithm fulfills this role of a marker in a non- FIFO system by using a coloring scheme on computation messages that works as follows:  Every process is initially white and turns red while taking a snapshot. The equivalent of the “Marker Sending Rule” is executed when a process turns red.  Every message sent by a white (red) process is colored white (red) indicating if it was sent before(after) snapshot.  Each process (which is initially white) becomes red as soon as it receives a red message for the first time and starts a virtual broadcast algorithm to ensure that all processes will eventually become red.
  • 17.
    Determining Messages in transit   White process records history of white msgs sent/received on each channel.  When a process turns red, it sends these histories along with its snapshot to the initiator process that collects the global snapshot.  Initiator process evaluates transit(LSi , LSj ) to compute state of a channel Cij :  SCij = white messages sent by pi on Cij − white messages received by pj on Cij = {send(mij )|send(mij ) ∈ LSi } − {rec(mij )|rec(mij ) ∈ LSj }.
  • 18.
    The Proposed Algorithm  Assumptions:  Author assumes a pre-established spanning tree on the distributed system for an initiator to perform a one-to-all broadcast to inform all other processes the initiation of a new global snapshot.  In doing so, it can be ensured that even without receiving any red message, a white process will learn that one snapshot execution has been initiated and then will execute the global-snapshot algorithm to construct the global snapshot cooperatively with all other processes
  • 19.
      There arethree phases in the algorithm  Phase 1: This is the phase of snapshot initiation. The initiator performs a one-to-all broadcast of INIT control messages along the pre-established spanning tree to inform all other processes .
  • 20.
      Phase 2:This is the phase of accumulating numbers of white messages. Upon the receipt of an INIT control message or a red computation message, a white process turns red and records its local state.  Subsequently, the algorithm directs the whole processes to calculate in a symmetrical manner the total number of white messages that each process is supposed to receive.  In particular, every process pi maintains a vector of size N, wmsg senti[], to count in wmsg senti[j] the number of white messages that it has sent to another process pj .
  • 21.
  • 22.
      Phase 3:The is the phase of recording channel states. When a red process pi receives a white computation message along a channel, such a message is added to the state of the channel. In addition, when the second phase is done and all white messages supposed to be received by pi have been received by pi, i.e. wmsg receivedi = sum wmsgi[i], pi turns white and then terminates the algorithm locally.
  • 23.
    Algorithm  Initiator - 1 3 2 4 1
  • 24.
    Initiator - 1 3 2 4 1 Control Messages & a Red message sent
  • 25.
    Initiator - 1 3 2 4 Red Message Received at 2 1
  • 26.
    Initiator - 1 3 sends white 3 Ctrl Mesg from Mesg to 1 1 reaches 3 2 4 1
  • 27.
    Initiator - 1 3 2 4 Red Ctrl Mesg 1 received White Mesg received at 1
  • 28.
    Summary   Global State detection difficult in Distributed Systems  Snapshot algorithm may not give an actual state but is very helpful in detecting Stable Properties
  • 29.
    References  [1] Jichiang Tsai, “Flexible Symmetrical Global-Snapshot Algorithms for Large-Scale Distributed Systems,” IEEE Transactions On Parallel And Distributed Systems, vol. Xx, no. Y, apr. 2012 [2] R. Garg, V. Garg, and Y. Sabharwal, “Efficient algorithms for global snapshots in large distributed systems,” IEEE Trans. Parallel and Distributed Systems, vol. 21, no. 5, pp. 620–630, May 2010. [3] A. D. Kshemkalyani, “Fast and message-efficient global snapshot algorithms for large-scale distributed systems,” IEEE Trans. Parallel and Distributed Systems, vol. 21, no. 9, pp. 1281–1209, Sept. 2010. [4]F. Mattern, “Efficient Algorithms for Distributed Snapshots and Global Virtual Time Approximation,” J. Parallel and Distributed Computing, pp. 423-434, Aug. 1993. [5] K.M. Chandy and L. Lamport, “Distributed Snapshots: Determining Global States of Distributed Systems,” ACM Trans. Computer Systems, vol. 3, no. 1, pp. 63-75, Feb. 1985.
  • 30.