Flexible Paxos
Quorum Intersection Revisited
Part I - Basic Paxos
Basic Paxos
● Three Roles:
○ Proposer: a process who wishes to have a particular value chosen
○ Acceptor: a process which agrees and persists decided values or the learner
○ Learner: a process wishing to learn the decided value
● Two Phase Process:
○ Prepare / Promise
○ Accept / Commit
Prepare / Promise
● A proposer selects a unique proposal number p and sends prepare(p) to the acceptors
● Each acceptor receives prepare(p)
○ If p is the highest proposal number promised, then p is written to persistent
storage and the acceptor replies with promise(p’,v’)
○ (p’,v’) is the last accepted proposal (if present) where p’ is the proposal number
and v’ is the corresponding proposed value
● Once the proposer receives promise from the majority of acceptors, it proceeds to
phase two. Otherwise, it may try again with higher proposal number
Accept / Commit
● Proposer selects a value v
○ If any proposal was returned in phase 1 then it must choose the value associated
with the highest proposal number.
○ If no proposals were returned, then the proposer can choose its own value for v
● The proposer then sends propose(p,v) to the acceptors
● Each acceptor receives a propose(p,v)
○ If p is equal to or greater than the highest promised proposal number, then the
promised proposal number and accepted proposal is written to persistent
storage and the acceptor replies with accept(p)
● Once the proposer receives accept(p) from the majority of acceptors, it learns that the
value v is decided
Basic Paxos - Normal
OperationP: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P:
C:
X: 1
P:
C:
X: 2
P:
C:
X: 3
Acceptor Acceptor
Proposer, Acceptor
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P:
C:
P:
C:
P: 10
C:
X: 3
Acceptor Acceptor
Proposer, Acceptor
Prepare(10) Prepare(10)
Basic Paxos - Normal
Operation
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C:
P: 10
C:
P:
C:
X: 3
Acceptor Acceptor
Proposer, Acceptor
Promise( _, _ ) Promise( _, _ )
Basic Paxos - Normal
Operation
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
3 3
3
P: 10
C: (10 ,3)
P: 10
C: (10, 3)
P: 10
C: (10, 3)
X: 3
Acceptor Acceptor
Proposer, Acceptor
Commit(10, 3) Commit(10, 3)
Basic Paxos - Normal
Operation
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C:
P: 10
C:
P: 10
C: (10, 3)
X: 3
Acceptor Acceptor
Proposer, Acceptor
Commit(10, 3) Commit(10, 3)
Basic Paxos - Normal
Operation
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C:
X: 1
P: 10
C:
P:
C:
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 1
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C:
X: 1
P: 10
C:
P:
C:
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 1
Papare(9 )
Papare( 9 )
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C:
X: 1
P: 10
C:
P:
C:
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 1
Reject
Reject
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C:
X: 1
P: 10
C:
P: 10
C:
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 2
Papare(11 )
Papare( 11 )
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 11
C:
X: 1
P: 11
C:
P: 11
C:
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 2
Promise( _, _ )
Promise( _, _ )
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 11
C:
X: 1
P: 11
C:
P: 11
C:
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 2
Commit(11, 1)
Commit(11, 1)
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
1 1
1
P: 11
C: (11, 1)
X: 1
P: 11
C: (11, 1)
P: 11
C: (11, 1)
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 2
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
1 1
1
P: 11
C: (11, 1)
X: 1
P: 11
C: (11, 1)
P: 11
C: (11, 1)
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 3
Prepare(12) Prepare(12)
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
1 1
1
P: 12
C: (11, 1)
X: 1
P: 12
C: (11, 1)
P: 11
C: (11, 1)
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 3
Promise(11, 1) Promise(11, 1)
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
1 1
1
P: 12
C: (11, 1)
X: 1
P: 12
C: (11, 1)
P: 11
C: (11, 1)
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 3
Commit(12, 1) Commit(12, 1)
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
1 1
1
P: 12
C: (12, 1)
X: 1
P: 12
C: (12, 1)
P: 11
C: (12, 1)
X: 3
Proposer, Acceptor Acceptor
Proposer, Acceptor
Basic Paxos - Conflict 3
Insight
● Phase 1 is not specific to the request so can be done before the request arrives and
can be reused
○ i.e., elect a leader, and then let it to propose all following proposals
Part II - Flexible Paxos
● It is only necessary for phase 1 quorums (Q1) and phase 2 quorums (Q2) to intersect
○ no need to require that Q1’s intersect with each other nor Q2’s intersect with
each other
○ Q1 + Q2 > N (N is the number of acceptors)
Observation
Example
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C: (10, 1)
X: 1
P:
C:
X: 2
P: 10
C:
Proposer, Acceptor Proposer, Acceptor
Acceptor
? ?
Acceptor Acceptor
Q1: 4
Q2: 2
P: 10
C:
P: 10
C: (10, 1)
Example
P: the highest promised proposal number
C: the last accepted proposal
X: the value a proposer want to propose
? ?
?
P: 10
C: (10, 1)
X: 1
P:
C:
X: 2
P: 10
C:
Proposer, Acceptor Proposer, Acceptor
Acceptor
? ?
Acceptor Acceptor
Q1: 4
Q2: 2
P: 10
C:
P: 10
C: (10, 1)
Prepare(11)
Prepare(11)
Prepare(11)
Implications
● Majority Quorums
● Simple Quorums
● Grid Quorums
Implications - Majority Quorums
● Majority Quorums: use quorums of size N/2 + 1
● When the number of acceptors N is even
○ We can safely reduce the size of Q2 by one from N/2 + 1 to N/2
Implications - Simple Quorums
● Simple Quorums: any acceptor is able to participate in a quorum and each
acceptor’s participation is counted equally
○ a straightforward generalization of majority quorums
● Paxos requires that all quorums intersect
● In FPaxos (Flexible Paxos) simple quorums require only that quorums from different
phases intersect
○ Q1 + Q2 > N (N is the number of acceptors)
○ for a given size of Q2 and number of acceptors N, the minimum size of our
first phase quorum is Q1 = N − Q2 + 1
Implications - Grid Quorums
● Grid Quorums: arrange the N nodes into a matrix of N1 columns by N2 rows,
where N1 ×N2 = N
● Paxos requires that all quorums intersect
○ then it requires one row and one column to form a quorum
● In FPaxos, we can safely reduce our quorums to one row of size N1 for Q1 and one
column of size N2 for Q2
Paxos FPaxos
Benchmark
● 64 bytes per request
● A single linux VM with a single core and 1GB of RAM
● use mininet to simulate a 10 Mbps network with 20 ms round trip time
Any Questions ?
Thank You!

Paper Reading: Flexible Paxos

  • 1.
  • 2.
    Part I -Basic Paxos
  • 3.
    Basic Paxos ● ThreeRoles: ○ Proposer: a process who wishes to have a particular value chosen ○ Acceptor: a process which agrees and persists decided values or the learner ○ Learner: a process wishing to learn the decided value ● Two Phase Process: ○ Prepare / Promise ○ Accept / Commit
  • 4.
    Prepare / Promise ●A proposer selects a unique proposal number p and sends prepare(p) to the acceptors ● Each acceptor receives prepare(p) ○ If p is the highest proposal number promised, then p is written to persistent storage and the acceptor replies with promise(p’,v’) ○ (p’,v’) is the last accepted proposal (if present) where p’ is the proposal number and v’ is the corresponding proposed value ● Once the proposer receives promise from the majority of acceptors, it proceeds to phase two. Otherwise, it may try again with higher proposal number
  • 5.
    Accept / Commit ●Proposer selects a value v ○ If any proposal was returned in phase 1 then it must choose the value associated with the highest proposal number. ○ If no proposals were returned, then the proposer can choose its own value for v ● The proposer then sends propose(p,v) to the acceptors ● Each acceptor receives a propose(p,v) ○ If p is equal to or greater than the highest promised proposal number, then the promised proposal number and accepted proposal is written to persistent storage and the acceptor replies with accept(p) ● Once the proposer receives accept(p) from the majority of acceptors, it learns that the value v is decided
  • 6.
    Basic Paxos -Normal OperationP: the highest promised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: C: X: 1 P: C: X: 2 P: C: X: 3 Acceptor Acceptor Proposer, Acceptor
  • 7.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: C: P: C: P: 10 C: X: 3 Acceptor Acceptor Proposer, Acceptor Prepare(10) Prepare(10) Basic Paxos - Normal Operation
  • 8.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: P: 10 C: P: C: X: 3 Acceptor Acceptor Proposer, Acceptor Promise( _, _ ) Promise( _, _ ) Basic Paxos - Normal Operation
  • 9.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose 3 3 3 P: 10 C: (10 ,3) P: 10 C: (10, 3) P: 10 C: (10, 3) X: 3 Acceptor Acceptor Proposer, Acceptor Commit(10, 3) Commit(10, 3) Basic Paxos - Normal Operation
  • 10.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: P: 10 C: P: 10 C: (10, 3) X: 3 Acceptor Acceptor Proposer, Acceptor Commit(10, 3) Commit(10, 3) Basic Paxos - Normal Operation
  • 11.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: X: 1 P: 10 C: P: C: X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 1
  • 12.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: X: 1 P: 10 C: P: C: X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 1 Papare(9 ) Papare( 9 )
  • 13.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: X: 1 P: 10 C: P: C: X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 1 Reject Reject
  • 14.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: X: 1 P: 10 C: P: 10 C: X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 2 Papare(11 ) Papare( 11 )
  • 15.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 11 C: X: 1 P: 11 C: P: 11 C: X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 2 Promise( _, _ ) Promise( _, _ )
  • 16.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 11 C: X: 1 P: 11 C: P: 11 C: X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 2 Commit(11, 1) Commit(11, 1)
  • 17.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose 1 1 1 P: 11 C: (11, 1) X: 1 P: 11 C: (11, 1) P: 11 C: (11, 1) X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 2
  • 18.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose 1 1 1 P: 11 C: (11, 1) X: 1 P: 11 C: (11, 1) P: 11 C: (11, 1) X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 3 Prepare(12) Prepare(12)
  • 19.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose 1 1 1 P: 12 C: (11, 1) X: 1 P: 12 C: (11, 1) P: 11 C: (11, 1) X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 3 Promise(11, 1) Promise(11, 1)
  • 20.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose 1 1 1 P: 12 C: (11, 1) X: 1 P: 12 C: (11, 1) P: 11 C: (11, 1) X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 3 Commit(12, 1) Commit(12, 1)
  • 21.
    P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose 1 1 1 P: 12 C: (12, 1) X: 1 P: 12 C: (12, 1) P: 11 C: (12, 1) X: 3 Proposer, Acceptor Acceptor Proposer, Acceptor Basic Paxos - Conflict 3
  • 22.
    Insight ● Phase 1is not specific to the request so can be done before the request arrives and can be reused ○ i.e., elect a leader, and then let it to propose all following proposals
  • 23.
    Part II -Flexible Paxos
  • 24.
    ● It isonly necessary for phase 1 quorums (Q1) and phase 2 quorums (Q2) to intersect ○ no need to require that Q1’s intersect with each other nor Q2’s intersect with each other ○ Q1 + Q2 > N (N is the number of acceptors) Observation
  • 25.
    Example P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: (10, 1) X: 1 P: C: X: 2 P: 10 C: Proposer, Acceptor Proposer, Acceptor Acceptor ? ? Acceptor Acceptor Q1: 4 Q2: 2 P: 10 C: P: 10 C: (10, 1)
  • 26.
    Example P: the highestpromised proposal number C: the last accepted proposal X: the value a proposer want to propose ? ? ? P: 10 C: (10, 1) X: 1 P: C: X: 2 P: 10 C: Proposer, Acceptor Proposer, Acceptor Acceptor ? ? Acceptor Acceptor Q1: 4 Q2: 2 P: 10 C: P: 10 C: (10, 1) Prepare(11) Prepare(11) Prepare(11)
  • 27.
    Implications ● Majority Quorums ●Simple Quorums ● Grid Quorums
  • 28.
    Implications - MajorityQuorums ● Majority Quorums: use quorums of size N/2 + 1 ● When the number of acceptors N is even ○ We can safely reduce the size of Q2 by one from N/2 + 1 to N/2
  • 29.
    Implications - SimpleQuorums ● Simple Quorums: any acceptor is able to participate in a quorum and each acceptor’s participation is counted equally ○ a straightforward generalization of majority quorums ● Paxos requires that all quorums intersect ● In FPaxos (Flexible Paxos) simple quorums require only that quorums from different phases intersect ○ Q1 + Q2 > N (N is the number of acceptors) ○ for a given size of Q2 and number of acceptors N, the minimum size of our first phase quorum is Q1 = N − Q2 + 1
  • 30.
    Implications - GridQuorums ● Grid Quorums: arrange the N nodes into a matrix of N1 columns by N2 rows, where N1 ×N2 = N ● Paxos requires that all quorums intersect ○ then it requires one row and one column to form a quorum ● In FPaxos, we can safely reduce our quorums to one row of size N1 for Q1 and one column of size N2 for Q2 Paxos FPaxos
  • 31.
    Benchmark ● 64 bytesper request ● A single linux VM with a single core and 1GB of RAM ● use mininet to simulate a 10 Mbps network with 20 ms round trip time
  • 32.