Presentation on how to chat with PDF using ChatGPT code interpreter
Byzantine ordered consensus
1. Yongrae Jo
Pohang University of Science and Technology
System Software Laboratory
Byzantine Ordered Consensus without
Byzantine Oligarchy
Yunhao Zhang, Cornell University; Srinath Setty, Qi Chen, and Lidong Zhou, Microsoft Research;
Lorenzo Alvisi, Cornell University
14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2020)
2. 2
State Machine Replication (SMR)
State Machine Replication
b c a e d
n n+1
n-1
n-2 n+2
Sequence#
Commands
b c d e
a
The SMR guarantees consistent states by executing
totally ordered sequence of commands
Set of commands
Total-ordered commands
3. 3
State Machine Replication (SMR)
Leader
b c d e
a
Set of commands
Replica Replica Replica
Can control which commands
are processed first
Replica
b c a e d
n n+1
n-1
n-2 n+2
Order 1
e d b c a
Order 2
a c e d b
Order 3
Depending on the leader’s choice,
different total order made possible
4. 4
Importance of Transaction Order
Ledger Ledger
Ledger Ledger
Order does not matter
for fault-tolerance
(as long as consistency is guaranteed)
…
자본 시장의 첨단을 달리는 미국 월스트
리트에서도 이와 비슷한 일이 벌어지고
있다. 이른바 ‘초단타매매(HTF·
High
Freqeuncy Trading)’다. 이런 초미세 시간
차 매매로 수익을 올리는 어둠의 약탈자
들이 존재한다는 사실은 비교적 최근에
알려졌다. 이들은 1000분의 1초(밀리세컨
드) 단위 속도로 진행되는 일종의 선행매
매(Front running)를 통해 막대한 이익을
챙긴다. 규모도 정확히 파악하기 어려울
정도다. 주로 누군가의 주문을 먼저 보고
같은 주문을 미리 내 가격을 올려 파는 방
식을 구사한다. … ([북리뷰] 플래시 보이
스, 조선비즈)
Traditional BFT Blockchain
https://biz.chosun.com/site/data/html_dir/2014/10/10/2014101002481.html
Order matters
for financial transactions
5. 5
BFT-SMR Specification
⚫ Safety
• All non-faulty replicas agree on a total order for the execution of comman
ds despite Byzantine failures
⚫ Liveness
• Clients eventually receive replies to their requests
• i.e., a system should make progress despite Byzantine failures
6. 6
BFT-SMR Specification
⚫ Safety
• All non-faulty replicas agree on a total order for the execution of comman
ds despite Byzantine failures
⚫ Liveness
• Clients eventually receive replies to their requests
• i.e., a system should make progress despite Byzantine failures
b c a e d
n n+1
n-1
n-2 n+2
Order 1
e d b c a
Order 2
a c e d b
Order 3
Order itself does not matter so long as consistency is guaranteed !
Which order is
the most desirable ?
7. 7
Motivation
⚫ Safety
• All non-faulty replicas agree on a total order for the execution of comman
ds despite Byzantine failures
⚫ Liveness
• Clients eventually receive replies to their requests
• i.e., a system should make progress despite Byzantine failures
b c a e d
n n+1
n-1
n-2 n+2
Order 1
e d b c a
Order 2
a c e d b
Order 3
Order itself does not matter so long as consistency is guaranteed !
Is it possible to formally define an ordering property
to put a constraint on the order of commands?
Which order is
the most desirable ?
8. 8
Contribution
⚫ A new BFT specification: Byzantine ordered consensus
• Expand BFT-SMR correctness specification to express ordering
requirement rigorously
⚫ A new BFT architecture: Factoring ordering out of consensus
• Consensus-agnostic ordering phase
• Preprocessing for locking a position in the ledger
⚫ Design, implement, and evaluate Pompe
• BFT-SMR protocol based on Byzantine ordered consensus
Ordering Consensus
HotStuff, SBFT, …
Locks #n-th entry
in the ledger
Consensus on the
#n-th entry
in the ledger
9. 9
Byzantine ordered consensus : Concepts introduced
A consensus, in which any node can propose
a command with an ordering preference that
allows each node to express how the
command is ordered, while minimizing
Byzantine influence
Byzantine Ordered Consensus
10. 10
Byzantine ordered consensus : Concepts introduced
Politics of Byzantium
Byzantine Oligarchy
Byzantine Democracy
Byzantine nodes can
fully control order of
commands
Byzantine nodes can only have a
limited effect on the order of
commands
= The ordering preferences of
the honest nodes appear in the
final ledger.
+ Free will
https://thereader.mitpress.mit.edu/american-
democracy-is-less-broken-than-you-think/
https://issuetoktok1177.tistory.com/18
11. 11
Terms & Notations
Term Notation Definition
Command c A request to be executed
Ordering Indicator o
A metadata that represents a node’s ordering
preference
Proposal (o,c) A node’s proposal request with an ordering preference
Order-before
relation
<o
A comparison operator on two proposals.
i.e., For (o1, c1), (o2, c2), if o1 <o o2, then c1 before c2
Profile
Pi A set of proposal from node i
PC A set of proposal from all correct nodes
Trace -
An execution sequence of a profile by following the
ordering preference.
12. 12
Terms & Notations
Term Notation Definition
Command c A request to be executed
Ordering Indicator o
A metadata that represents a node’s ordering
preference
Proposal (o,c) A node’s proposal request with an ordering preference
Order-before
relation
<o
A comparison operator on two proposals.
i.e., For (o1, c1), (o2, c2), if o1 <o o2, then c1 before c2
Profile
Pi A set of proposal from node i
PC A set of proposal from all correct nodes
Trace -
An execution sequence of a profile by following the
ordering preference.
E.g., timestamp, dependency set/graph, sequence number
13. 13
Byzantine Oligarchy
For all profiles of PC,
For all pairs of commands (c1, c2) in PC,
There exists,
a trace for PC that results in c1 <o c2 , and
a trace for PC that results in c2 <o c1
in the ledgers of correct nodes
Byzantine nodes can determine the ordering of any two commands, regardl
ess of the ordering indicators from correct nodes.
Meaning
A protocol is subject to a Byzantine oligarchy if and only if …
14. 14
Free Will
For all profiles of PC,
1) There exists,
a trace for PC that all commands in PC appear
in the ledgers of correct nodes
2) There exists,
two profiles PA, PB that
for all commands c1, c2 that appear in both profiles,
there exists a trace for PA that results in c1 <o c2
there exists a trace for PB that results in c2 <o c1
A protocol respects the nodes’ free will, if and only if ….
15. 15
Byzantine Democracy
There exists, a profile of PC that
for all pairs of commands c1 and c2 ,
there exists a trace for PC that results in c1 <o c2 and
there exists a trace for PC that results in c2 <o c1
in the ledgers of correct nodes
a protocol upholds Byzantine democracy if and only if …
16. 16
If Politics of Byzantium allows Free will, then Byzantine democracy
17. 17
If Politics of Byzantium allows Free will, then Byzantine democracy
Byzantine Oligarchy
Byzantine Democracy
Byzantine nodes can
fully control order of
commands
+ Free will
(of correct nodes)
Theorem 3.1 places fundamental limits to the degree to
which it is possible to curb the influence of Byzantine nodes.
Byzantine nodes can only have a
limited effect on the order of
commands
= The ordering preferences of
the honest nodes appear in the
final ledger.
18. 18
Fundamental Limitation
⚫ In real-world scenario, we cannot distinguish which one is Byzantine or
not.
⚫ Therefore, we cannot completely eliminate Byzantine node’s influence
from their order manipulation
https://www.youtube.com/watch?v=m2wVye5FCS8&ab_channel=USENIX
19. 19
Ordering properties (1) : Ordering Unanimity
Ordering Unanimity
Node1 prefers c1 < c2 < c3
Node2 prefers c1 < c3 < c2
Node3 prefers c3 < c1 < c2
c1 ... c2
…
…
express ordering preference from majority of nodes
If all correct nodes prefer c1 < c2, then c1 < c2 in the output ledger
22. 22
Ordering properties (2) : Ordering Linearizability
Ordering Linearizability
Escape the Condorcet cycle trap
→ real-time happened before, a relation that is inherently acyclic
express ordering preference as timestamp
Cmd 1 Cmd 2
Highest timestamp Lowest timestamp
If all correct nodes confirms that cmd1’s highest
timestamp is lower than cmd2’s lowest timestamp,
then cmd1 is ordered before cmd2
Collect all correct node’s timestamp for cmd1
23. 23
Pompe
⚫ A new protocol designed for Byzantine ordered consensus
⚫ A cleanly separate ordering phase for Byzantine ordered consensus and
prevents Byzantine oligarchies by enforcing ordering linearizability
⚫ Two phase architecture: Ordering / (modular) Consensus phase
Ordering phase Consensus phase
Node
1
<Request
TS, c>
<ResponseTS,
c, ts>
<Sequence,
c, T>
<SequenceResponse, ack, h>
Node
2
Node
3
Node
4
Ordering
Linearizability
HotStuff
SBFT
Locks #n-th entry in the ledger
In a way that Byzantine nodes cannot
alter
Reach consensus on a stable prefix of the final
sequence, following the total ordering decisions in the
ordering phase
24. 24
System Model
⚫ A distributed system with a set of n = 3f + 1, where up to f nodes can
be Byzantine
⚫ Standard cryptographic primitives
⚫ Each node holds a private / public key to digitally sign / verify
messages
⚫ An adversarial network that can drop, reorder, or delay messages, but
weak synchrony for liveness
⚫ Each node has access to a timer, which produces monotonically
increasing timestamps each time it is queried.
25. 25
Pompe: Ordering Phase
⚫ Uses timestamps as ordering indicators.
⚫ Two steps
• 1. Pick the median timestamp to exclude Byzantine timestamps
• 2. Determine ledger position based on median timestamp
Node 1
<RequestTS, c>
<ResponseTS, c, ts>
<Sequence, c, T>
<SequenceResponse, ack, h>
Node 2
Node 3
Node 4
Wait for
2f+1 timestamps
→ Pick the median
Wait for
2f+1 responses (quorum)
→ Determine c’s position (k)
in the ledger
h: (hash of sequence)
26. 26
Pompe: Consensus Phase
⚫ Ensure that all correct nodes reach consensus on a certain prefix of
the total order constructed in the ordering phase
⚫ Can employ any standard leader-based BFT SMR protocol
<Collect, k>
<CollectResponse, k, S>
Leader-based
BFT SMR
(e.g., HotStuff)
Node 1
Node 2
Node 3
Node 4
k: consensus slot (ledger position)
28. 28
Experimental Setup
⚫ Deploy nodes at Azure cloud platform
• 100 Standard D16s_v3 (16 vcpus, 64 GB memory) VMs
• Each running Ubuntu Linux 18.04
• 34 in West US, 33 in SouthEast Asia, and 33 in North Europe
⚫ Experiments only for failure-free executions
⚫ Clients that submit their commands in a closed loop
• Submits a next command after receiving a response
⚫ Benchmark that writes a random value to a randomly selected key in a
key-value store.
⚫ Performance metrics
• client-perceived latency (measured in ms)
• Ordering latency / Consensus latency
• throughput (in commands/second).
29. 29
Evaluation: Pompe at Single Datacenter
Batch size of consensus phase
Batch size of ordering phase
Pompe’s separation of ordering / consensus phase enables higher batching opportunities
Pompe has
higher latency
than its baseline
four-node configuration in single datacenter
50ms
58.8ms
32. 32
Conclusion
⚫ There is a fundamental gap between the traditional SMR correctness
specification and the threat from order manipulation in blockchain
⚫ Proposes, a new primitive, Byzantine Ordered Consensus, that allows
rigorous expression and efficient enforcement of ordering
requirements
⚫ Design a modular architecture for Byzantine ordered consensus and
built Pompe which enforces ordering linearizability with performance
comparable to state-of-the-art systems
35. 35
Byzantine Oligarchy
Definition. Byzantine Oligarchy
A protocol execution is subject to a Byzantine oligarchy if and only if, for all profiles of
correct nodes PC, for all pairs of commands c1 and c2 in PC, there exists a trace for PC
that results in c1 before c2 in the ledgers of correct nodes and another trace for PC
that results in c2 before c1 in the ledgers of correct nodes.
36. 36
Free will & Byzantine Democracy
Definition. Byzantine Democracy
We say that a protocol upholds Byzantine democracy if and only if there exists a
profile of correct nodes PC, such that for all pairs of commands c1 and c2 in PC, there
exists a trace for PC that results in c1 before c2 in the ledgers of correct nodes and
another trace for PC that results in c2 before c1 in the ledgers of correct nodes.
Definition. Free will
We say that a protocol respects the nodes’ free will if and only if (i) for all profiles of
correct nodes PC , there exists a trace for PC, such that all commands in PC appear in
the ledgers of correct nodes in the trace and (ii) there exist two profiles PA and PB,
such that, for all commands c1 and c2 that appear in both profiles, there exists a trace
for PA that results in c1 before c2 in the ledgers of correct nodes and there exists a
trace for PB that results in c2 before c1 in the ledgers of correct nodes.
