2. Basics of Message Delivery
▸sendp(m)
▹Transmission of message m by process p to a set of
destinations denoted destinations(m)
▸receiveq(m)
▹Reception of message m by process q
▹sendp(m) receiveq(m)
▸deliverq(m)
▹Delivery of message m to process q
▹receiveq(m) deliverq(m)
Theoretical Foundations > Causal Delivery
2
3. Causal Delivery
▸Causal Delivery:
▹If sendi(w) sendj(m), and
q ∈ destinations(w) and q ∈ destinations(m),
then deliverq(w) deliverq(m)
▸What’s an example of a causal delivery protocol?
▹Transmission Control Protocol (TCP)
Theoretical Foundations > Causal Delivery
3
4. Broadcast-based Causal Delivery
▸Broadcast:
▹when a process sends a message, it sends the
message to every process in the system
▹Called a multicast when sending to a defined group
Theoretical Foundations > Causal Delivery > Broadcast-based
4
5. BSS Protocol
▸Broadcast-based causal delivery protocol
▹Assumes an external service implements group
abstraction
▹sendp(m) broadcasts message m to the entire group in a
single action
▸Assumes a lossless network
▹Any message broadcast will eventually be received unless
the sender or destination fail
▸Assumes a failure detection mechanism exists
▹Will remove failed processes from the group
▹Will flush broadcasts at the time of a failure
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
5
6. BSS Protocol Rules
▸Rule 1
Before sending message m, process i increments
Ci[i] and timestamps m.
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
6
7. BSS Protocol Rules
▸Rule 2
On reception of message m sent by process i and
timestamped tm, process j (≠ i) delays delivery of
m until:
∀k:[1…n]
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
7
tm[k] = Cj[k] + 1; if k = i
tm[k] ≤ Cj[k]; if k ≠ i
8. BSS Protocol Rules
▸Rule 3
When a message m is delivered, Cj is updated to
max(Cj, tm).
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
8
9. BSS Safety and Liveness
▸Safety:
▹messages are always delivered in causal order
▹Causal delivery is never violated
▸Liveness:
▹a message will never be indefinitely delayed
▹Every message will be delivered eventually
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
9
10. BSS Protocol Exercise
▸Rule 1: i increments Ci[i] and timestamps m
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
10
P
Q
R
[0 0 0]
[0 0 0]
[0 0 0]
11. BSS Protocol Exercise
▸Rule 1: i increments Ci[i] and timestamps m
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
11
P
Q
R
[1 0 0]
[0 0 0]
[0 0 0]
[1 0 0]
29. BSS Protocol Exercise
▸Rule 3: Cj is updated to max(Cj, tm)
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
29
P
Q
R
[1 1 1]
[1 1 1]
[1 1 1]
[1 0 0]
[0 0 1]
[1 1 1]
[1 1 1]
30. Proof of BSS Safety
▸Safety:
▹messages are always delivered in causal order
▹Causal delivery is never violated
▸Must prove
▹If send(m1) send(m2)
then deliveri(m1) deliveri(m2)
▹Two cases:
1. Same process: sendp(m1) sendp(m2)
2. Different processes: sendp(m1) sendq(m2)
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
30
31. Proof of BSS Safety: Case 1
▸If sendp(m1) sendp(m2) then deliveri(m1) deliveri(m2)
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
31
P
Q
R
[2 1 0]
[2 1 0]
[2 1 0]
[1 0 0] [2 0 0]
[1 1 0]
[1 1 0]
32. Proof of BSS Safety: Case 2
▸If sendp(m1) sendq(m2) then deliveri(m1) deliveri(m2)
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
32
X
Y
Z
create(R1)
create(R1)
update(R1)
R1
R1
R1
R1
R1
[1 1 0]
[1 1 0]
[1 1 0]
[0 1 0]
[0 1 0]
[1 1 0]
R1
33. Quiz Question
▸Assume sentQ(m) sentQ(n),
process P ∈ dests(m), and P ∈ dests(n).
If deliverP(n) deliverP(m), then safety is
violated.
▹True
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
33
34. Proof of BSS Liveness
▸Liveness:
▹a message will never be indefinitely delayed
▹Every message will be delivered eventually
▸Must prove
▹If sendi(m) and receivej(m) then deliverj(m)
▹Two counterexamples:
1. If k = i then tm[k] will never equal Cj[k] + 1
2. If k ≠ i then tm[k] will always be greater than Cj[k]
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
34
35. Proof of BSS Liveness: Counterexample 1
▸If k = i then tm[k] will never equal Cj[k] + 1
▸tm[k] is not less than Cj[k] + 1
▹Process i sent message m
▹Hence, Cj[i] < Ci[i] when m was timestamped
▸If tm[k] is more than Cj[k] + 1
▹Process i sent other messages prior to m
▸BSS assumes a lossless network
▹Process j will eventually receive the other messages from i
▹Hence, tm[k] will eventually equal Cj[k] + 1
▸Proves if sendi(m) and receivej(m) then deliverj(m)
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
35
36. Proof of BSS Liveness: Counterexample 2
▸If k ≠ i then tm[k] will always be greater than Cj[k]
▸tm[k] is greater than Cj[k]
▹Before sending message m to process j,
process i received messages from process k that process j
has not received
▸BSS is broadcast-based
▹Process k also sent these messages to process j
▸BSS assumes a lossless network
▹Process j will eventually receive the messages from k
▹Hence, tm[k] will eventually be less than or equal to Cj[k]
▸Proves if sendi(m) and receivej(m) then deliverj(m)
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
36
37. Quiz Question
▸Assume sentQ(m) sentQ(n),
process P ∈ dests(m), and P ∈ dests(n).
If deliverP(m) never occurs, then liveness is
violated.
▹True
Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol
37
40. Unicast-based Causal Delivery
▸Unicast:
▹when a process sends a message, it sends the
message to one process in the system
Theoretical Foundations > Causal Delivery > Unicast-based
40
41. RST Protocol
▸RST:
▹Raynal, Schiper & Toueg
▸A causal delivery protocol
▹Uses matrices to track messages sent
▸Assumes a lossless network
▹Any message unicast will eventually be received
unless the sender or destination fail
Theoretical Foundations > Causal Delivery > Unicast-based > RST Protocol
41
42. RST Protocol Variables
▸Every process manages two variables
▹DELIVj[i]
▸An array [1…n] that represents the number of messages
sent from process i and delivered to process j
▹SENTj[k, l]
▸A matrix [n x n] that represents the number of messages
sent from process k to process l (not necessarily delivered)
from process j’s perspective
▸Both are initialized with 0s
Theoretical Foundations > Causal Delivery > Unicast-based > RST Protocol
42
43. RST Protocol Rules
▸Rule 1
After sending message m to process j with a
copy of SENTi (called tm), process i increments
SENTi[i, j].
Theoretical Foundations > Causal Delivery > Unicast-based > RST Protocol
43
48. RST Protocol Rules
▸Rule 2
On reception of message m from process i and
piggybacked with tm, process j delays delivery
of m until:
∀k:[1…n] tm[k, j] ≤ DELIVj[k]
Theoretical Foundations > Causal Delivery > Unicast-based > RST Protocol
48
115. Proof of RST Liveness
▸Liveness:
▹a message will never be indefinitely delayed
▹Every message will be delivered eventually
▸Must prove
▹If sendi(m) and receivej(m) then deliverj(m)
▹Counterexample:
▸∃k such that tm[k, j] will always be greater than DELIVj[k]
Theoretical Foundations > Causal Delivery > Unicast-based > RST Protocol
115
116. Proof of RST Liveness: Counterexample
▸∃k such that tm[k, j] will always be greater than
DELIVj [k]
▸tm[k, j] is greater than DELIVj [k]
▹Process k sent messages to process j prior to message m
being sent to process j
▹Process j has not received these messages yet
▸RST assumes a lossless network
▹Process j will eventually receive the messages from k
▹tm[k, j] will eventually be less than or equal to DELIVj [k]
▸Proves if sendi(m) and receivej(m) then deliverj(m)
Theoretical Foundations > Causal Delivery > Unicast-based > RST Protocol
116
118. Quiz Question
▸Considering the BSS and RST algorithms, which
of the following is false?
▹The BSS algorithm is designed for multicast systems.
▹The RST algorithm is designed for unicast systems.
▹The BSS algorithm requires smaller messages.
▹The RST algorithm requires more messages.
Theoretical Foundations > Causal Delivery
118
119. Team Composition
▸ 5 teams with 4 (in one case 3) members
▸ All members are supposed to contributed equally in the project
▸ Good to schedule weekly meeting for discussion checking
progress
Team Introduction
119
Editor's Notes
Dotted line shows how Rule 2 delays the delivery to ensure safety property
What is the value of the SENT variable attached to m1?