Basic
Consensus Algorithms
Mar. 2020
Sangmoon Oh (halfface@chollian.net)
v1.0
2
목차
• Byzantine Fault
• Byzantine Fault Tolerance (BFT)
• 3m + 1 Processors Algorithm
• pBFT
• Proof of Work (PoW)
• Ethash
• PoW Finality Problem
• Casper FFG
3
II. Consensus AlgorithmByzantine Fault (Byzantine General Problem)
Enemy
attack
retreat
retreat
retreat
retreat
retreat
attack
attack
attack
attack
Agree on Attack/Retreat
Protocol or Algorithm
 SIFT: design and analysis of a fault-tolerant computer for aircraft control (1978)
 



4
II. Consensus AlgorithmByzantine Fault - Simple Example
2 Loyal Generals with 1 Unknown Traitor
1
3
2
Traitor
att
att
att
ret
att
att
(att,att,att)
Attack
(att,ret,att)
Attack
1
3
2
Traitor
att
ret
ret
ret
ret
ret
1
3
2
Traitor
att
att
att
ret
ret
ret
(ret,att,ret)
Retreat
(att,att,ret)
Attack
(ret,ret,ret)
Retreat
(att,ret,ret)
Retreat
Not Agreed !!
Case 1) Case 2) Case 3)
5
II. Consensus AlgorithmByzantine Fault Tolerance (BFT)
Enemy
General
Traitor
General Traitor
General
General
Attack
G
T
G T
G
G
Attack
Attack
Attack Retreat
G
T
G T
G
G
Retreat Retreat
Retreat
Algorithm 1
Retreat
Attack
G
T
G T
G
G
Attack
Attack
Retreat
G
T
G T
G
G
Retreat Retreat
RetreatAlgorithm 2
Always
Agreement
No Agreement
Possible
A1: BTF
A2: Not BTF
6
II. Consensus AlgorithmByzantine Fault Revisited
Enemy
 Synchronous or Asynchronous
Message Delivery
 Forgeable vs. Unforgeable Message
 Level of Connectivity
 Option Space
Is BTF Possible ?
What Algorithms ?
7
II. Consensus Algorithm3m + 1 Processors Algorithm - Overview
With m faulty processors,
Min. (3m +1) processors are needed
to tolerate the fault.
Algorithm OM(m)
n  3m + 1
General
Traitor General
General General
GeneralTraitor
General General
General
General
Traitor
Traitor
General General
General
General
Traitor
Traitor
General

m=1, n=3 (< 3m+1) m=1, n=4 ( 3m+1) m=2, n=6 (< 3m+1) m=2, n=7 ( 3m+1)
(Synchronized message delivery, Forgeable message, Full connected)
OM(1) OM(2)
8
II. Consensus Algorithm3m + 1 Processors Algorithm - Instructions
m nmin
Instruction
1 4 OM(0), OM(1)
2 7 OM(0), OM(1), OM(2)
3 10 OM(0), OM(1), OM(2), OM(3)









Recursive Relay
v:1:4
v:1:2:4
v:1:2
v:1:2:3 v:1:2:3:4
4
1
2
3
v:1:4
v:1:2:4
v:1:3:4
v:1:2:3:4
v:1:3:2:4
 1 + 2C1 + 2C11C1 = 5
v:1:::4
9
II. Consensus Algorithm3m + 1 Processors Algorithm - Simplest Case
Case v:2:1 v:2:3:1 v:2:4:1 v:2:::1 v:2:3 v:2:1:3 v:2:4:3 v:2:::3 v:2:4 v2:1:4 v:2:3:4 v:2:::4
1) A A R 2A + R A A R 2A + R R A A 2A + R
2) A R A 2A + R R A A 2A + R A A R 2A + R
3) A R R A + 2R R A R A + 2R R A R A + 2R
4) R A A 2A + R A R A 2A + R A R A 2A + R
5) R A R A + 2R A R R A + 2R R R A A + 2R
6) R R A A + 2R R R A A + 2R A R R A + 2R
10
II. Consensus Algorithm3m + 1 Processors Algorithm - Complexity
General General
GeneralTraitor
General General
General
General
Traitor
Traitor
General
Faulty Total Recursion Number of Message
1 4 OM(0), OM(1) 4*3 + 4*3*2
2 7 OM(0), OM(1), OM(2) 7*6 + 7*6*5 + 7*6*5*4
3 10 OM(0), OM(1), OM(2), OM(3) 10*9 + 10*9*8 + 10*9*8*7 + 10*9*8*7*6









m 3m + 1 OM(0), OM(1), , OM(m) (3m+1)*3m + (3m+1)*3m*(3m-1) +  + (3m+1)*3m2m
Complexity : O(nm)
General General
General
General
Traitor
Traitor
11
II. Consensus AlgorithmState Machine Replication
a : 100
b : 100
c : 100
a : 120
b : 110
c : 70
a : 90
b : 100
c : 110
tx1
tx3
tx4
tx2
tx1 :
b a, 20
tx2 :
b c, 10
tx3 :
c b, 30
tx4 :
a c, 30
a : 100
b : 100
c : 100
a : 120
b : 110
c : 70
a : 90
b : 100
c : 110
tx1
tx3
tx4
tx2
a : 100
b : 100
c : 100
a : 120
b : 110
c : 70
a : 90
b : 100
c : 110
tx1
tx3
tx4
tx2
Ordering Txs
Decentralized way
Communication and
Corporation
12
II. Consensus AlgorithmpBFT (Practical Byzantine Fault Tolerance)
1
2
3
4
client
Pre-prepare
(~ n)
Prepare
(~ n(n-1))
Commit
(~ n(n-1))
Primary Create and share next state
candidate
Backups Validate next state candidate
 Pre-prepare
Sender Share validated(accepted) next
state candidate
Receiver Collect the acceptance of the
other nodes
Commit if enough nodes
accepted ( >
2𝑛+1
3
)
 Prepare, Commit
tx
13
II. Consensus AlgorithmProof of Work (PoW)
4 3
2
1
5
 Compute
 Solve
 Send
 Send
 Send
 Send
 Compute
 Compute
 Compute
 Verify
 Verify
 Verify
 Verify
 Compute
Try a puzzle hard to solve but easy to verify.
 Competition
 Reward
14
II. Consensus Algorithm
4 3
2
1
5
 Compute
 Solve
 Send
 Send
 Send
 Send
 Compute
 Compute
 Compute
 Verify
 Verify
 Verify
 Verify
 Compute
Hash based PoW
Data + Nonce SHA-256(Data + Nonce)
Hello, world! 0x315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3
Hello, world!0 0x1312af178c253f84028d480a6adc1e25e81caa44c749ec81976192e2ec934c64
Hello, world!1 0xe9afc424b79e4f6ab42d99c81156d3a17228d6e1eef4139be78e948a9332a7d8
Hello, world!2 0xae37343a357a8297591625e7134cbea22f5928be8ca2a32aa475cf05fd4266b7
Hello, world!4248 0x6e110d98b388e77e9c6f042ac6b497cec46660deef75a55ebc7cfdf65cc0b965
Hello, world!4249 0xc004190b822f1669cac8dc37e761cb73652e7832fb814565702245cf26ebb9e6
Hello, world!4250 0x0000c3af42fc31103f1fdc0151fa747ff87349a4714df7cc52ea464e12dcd4e9
Puzzle: Find Nonce for SHA-256(Data, Nonce)
to Start with 0000
15
II. Consensus AlgorithmEthash - PoW Algorithm for Ethereum 1.0
 https://github.com/ethereum/wiki/wiki/Ethash
Block # Date Difficulty Target
9,000,000 2019-11-26 2,573,664,196,528,490 0x0000000000001bff809762e524...
8,000,000 2019-06-21 2,037,888,242,889,388 0x000000000000235be453641714...
6,000,000 2018-07-21 3,483,739,548,912,554 0x00000000000014af19378e94eb
3,000,000 2017-01-16 103,975,266,902,792 0x000000000002b506bb4c196d40
4 3
2
1
5
 Compute
 Solve
 Send
 Send
 Send
 Send
 Compute
 Compute
 Compute
 Verify
 Verify
 Verify
 Verify
 Compute
Keccak(header, transactions, nonce) = 0x0000...
def mine(full_size, dataset, header, difficulty):
target = zpad(encode_int(2**256 // difficulty), 64)[::-1]
from random import randint
nonce = randint(0, 2**64)
while hashimoto_full(full_size, dataset, header, nonce) > target:
nonce = (nonce + 1) % 2**64
return nonce
difficulty x target = 2256
16
II. Consensus AlgorithmPoW - Finality Problem
Solve Bm
Solve Bn
Bm
Two different miners may generate a block simultaneously.
Bm
Bn
Bn
17
II. Consensus AlgorithmPoW - Finality Problem
Solve Bm2
Solve Bn2
Bm
Another two different miners may generate a block simultaneously again.
Bm
Bn
Bn
Bn2
Bn2Bm2 Bm2
18
II. Consensus AlgorithmPoW - Finality Problem
Bm
Bn
Bm
Bn
Bm
Bn
Bm
Bn
Bn
  
  
txb(b a, 10)
txb(b a, 10)
19
II. Consensus AlgorithmCasper FFG - Finality System
 "Casper FFG" paper by Vitalik and Virgil
PoS based Finality System
which Overlays an existing
PoW blockchain.
 Accountability : Penalty
 Dynamic validators
 Modular overlay
: a partial consensus mechanism
20
II. Consensus AlgorithmCasper FFG - Process
1) Voting
 Vote for link(edge)
 2 Commandments
(Slashing conditions)
2) Justifying
  2/3 votes
3) Finalizing
 Canonical chain
1)
2)
3)
21
AppendixReferences
 Consensus
 Byzantine fault
 Reaching Agreement in the Presence of Faults (1980, M. PEASE, R, SHOSTAK, AND L. LAMPORT)
 The Byzantine Generals Problem (1982, LESLIE LAMPORT, ROBERT SHOSTAK, and MARSHALL PEASE)
 Practical Byzantine Fault Tolerance (1999, Miguel Castro and Barbara Liskov)
 pBFT— Understanding the Consensus Algorithm
 Proof-of-Work, Explained
 Hashcash
 Ethash

Basic Consensus Algorithms

  • 1.
    Basic Consensus Algorithms Mar. 2020 SangmoonOh (halfface@chollian.net) v1.0
  • 2.
    2 목차 • Byzantine Fault •Byzantine Fault Tolerance (BFT) • 3m + 1 Processors Algorithm • pBFT • Proof of Work (PoW) • Ethash • PoW Finality Problem • Casper FFG
  • 3.
    3 II. Consensus AlgorithmByzantineFault (Byzantine General Problem) Enemy attack retreat retreat retreat retreat retreat attack attack attack attack Agree on Attack/Retreat Protocol or Algorithm  SIFT: design and analysis of a fault-tolerant computer for aircraft control (1978)     
  • 4.
    4 II. Consensus AlgorithmByzantineFault - Simple Example 2 Loyal Generals with 1 Unknown Traitor 1 3 2 Traitor att att att ret att att (att,att,att) Attack (att,ret,att) Attack 1 3 2 Traitor att ret ret ret ret ret 1 3 2 Traitor att att att ret ret ret (ret,att,ret) Retreat (att,att,ret) Attack (ret,ret,ret) Retreat (att,ret,ret) Retreat Not Agreed !! Case 1) Case 2) Case 3)
  • 5.
    5 II. Consensus AlgorithmByzantineFault Tolerance (BFT) Enemy General Traitor General Traitor General General Attack G T G T G G Attack Attack Attack Retreat G T G T G G Retreat Retreat Retreat Algorithm 1 Retreat Attack G T G T G G Attack Attack Retreat G T G T G G Retreat Retreat RetreatAlgorithm 2 Always Agreement No Agreement Possible A1: BTF A2: Not BTF
  • 6.
    6 II. Consensus AlgorithmByzantineFault Revisited Enemy  Synchronous or Asynchronous Message Delivery  Forgeable vs. Unforgeable Message  Level of Connectivity  Option Space Is BTF Possible ? What Algorithms ?
  • 7.
    7 II. Consensus Algorithm3m+ 1 Processors Algorithm - Overview With m faulty processors, Min. (3m +1) processors are needed to tolerate the fault. Algorithm OM(m) n  3m + 1 General Traitor General General General GeneralTraitor General General General General Traitor Traitor General General General General Traitor Traitor General  m=1, n=3 (< 3m+1) m=1, n=4 ( 3m+1) m=2, n=6 (< 3m+1) m=2, n=7 ( 3m+1) (Synchronized message delivery, Forgeable message, Full connected) OM(1) OM(2)
  • 8.
    8 II. Consensus Algorithm3m+ 1 Processors Algorithm - Instructions m nmin Instruction 1 4 OM(0), OM(1) 2 7 OM(0), OM(1), OM(2) 3 10 OM(0), OM(1), OM(2), OM(3)          Recursive Relay v:1:4 v:1:2:4 v:1:2 v:1:2:3 v:1:2:3:4 4 1 2 3 v:1:4 v:1:2:4 v:1:3:4 v:1:2:3:4 v:1:3:2:4  1 + 2C1 + 2C11C1 = 5 v:1:::4
  • 9.
    9 II. Consensus Algorithm3m+ 1 Processors Algorithm - Simplest Case Case v:2:1 v:2:3:1 v:2:4:1 v:2:::1 v:2:3 v:2:1:3 v:2:4:3 v:2:::3 v:2:4 v2:1:4 v:2:3:4 v:2:::4 1) A A R 2A + R A A R 2A + R R A A 2A + R 2) A R A 2A + R R A A 2A + R A A R 2A + R 3) A R R A + 2R R A R A + 2R R A R A + 2R 4) R A A 2A + R A R A 2A + R A R A 2A + R 5) R A R A + 2R A R R A + 2R R R A A + 2R 6) R R A A + 2R R R A A + 2R A R R A + 2R
  • 10.
    10 II. Consensus Algorithm3m+ 1 Processors Algorithm - Complexity General General GeneralTraitor General General General General Traitor Traitor General Faulty Total Recursion Number of Message 1 4 OM(0), OM(1) 4*3 + 4*3*2 2 7 OM(0), OM(1), OM(2) 7*6 + 7*6*5 + 7*6*5*4 3 10 OM(0), OM(1), OM(2), OM(3) 10*9 + 10*9*8 + 10*9*8*7 + 10*9*8*7*6          m 3m + 1 OM(0), OM(1), , OM(m) (3m+1)*3m + (3m+1)*3m*(3m-1) +  + (3m+1)*3m2m Complexity : O(nm) General General General General Traitor Traitor
  • 11.
    11 II. Consensus AlgorithmStateMachine Replication a : 100 b : 100 c : 100 a : 120 b : 110 c : 70 a : 90 b : 100 c : 110 tx1 tx3 tx4 tx2 tx1 : b a, 20 tx2 : b c, 10 tx3 : c b, 30 tx4 : a c, 30 a : 100 b : 100 c : 100 a : 120 b : 110 c : 70 a : 90 b : 100 c : 110 tx1 tx3 tx4 tx2 a : 100 b : 100 c : 100 a : 120 b : 110 c : 70 a : 90 b : 100 c : 110 tx1 tx3 tx4 tx2 Ordering Txs Decentralized way Communication and Corporation
  • 12.
    12 II. Consensus AlgorithmpBFT(Practical Byzantine Fault Tolerance) 1 2 3 4 client Pre-prepare (~ n) Prepare (~ n(n-1)) Commit (~ n(n-1)) Primary Create and share next state candidate Backups Validate next state candidate  Pre-prepare Sender Share validated(accepted) next state candidate Receiver Collect the acceptance of the other nodes Commit if enough nodes accepted ( > 2𝑛+1 3 )  Prepare, Commit tx
  • 13.
    13 II. Consensus AlgorithmProofof Work (PoW) 4 3 2 1 5  Compute  Solve  Send  Send  Send  Send  Compute  Compute  Compute  Verify  Verify  Verify  Verify  Compute Try a puzzle hard to solve but easy to verify.  Competition  Reward
  • 14.
    14 II. Consensus Algorithm 43 2 1 5  Compute  Solve  Send  Send  Send  Send  Compute  Compute  Compute  Verify  Verify  Verify  Verify  Compute Hash based PoW Data + Nonce SHA-256(Data + Nonce) Hello, world! 0x315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3 Hello, world!0 0x1312af178c253f84028d480a6adc1e25e81caa44c749ec81976192e2ec934c64 Hello, world!1 0xe9afc424b79e4f6ab42d99c81156d3a17228d6e1eef4139be78e948a9332a7d8 Hello, world!2 0xae37343a357a8297591625e7134cbea22f5928be8ca2a32aa475cf05fd4266b7 Hello, world!4248 0x6e110d98b388e77e9c6f042ac6b497cec46660deef75a55ebc7cfdf65cc0b965 Hello, world!4249 0xc004190b822f1669cac8dc37e761cb73652e7832fb814565702245cf26ebb9e6 Hello, world!4250 0x0000c3af42fc31103f1fdc0151fa747ff87349a4714df7cc52ea464e12dcd4e9 Puzzle: Find Nonce for SHA-256(Data, Nonce) to Start with 0000
  • 15.
    15 II. Consensus AlgorithmEthash- PoW Algorithm for Ethereum 1.0  https://github.com/ethereum/wiki/wiki/Ethash Block # Date Difficulty Target 9,000,000 2019-11-26 2,573,664,196,528,490 0x0000000000001bff809762e524... 8,000,000 2019-06-21 2,037,888,242,889,388 0x000000000000235be453641714... 6,000,000 2018-07-21 3,483,739,548,912,554 0x00000000000014af19378e94eb 3,000,000 2017-01-16 103,975,266,902,792 0x000000000002b506bb4c196d40 4 3 2 1 5  Compute  Solve  Send  Send  Send  Send  Compute  Compute  Compute  Verify  Verify  Verify  Verify  Compute Keccak(header, transactions, nonce) = 0x0000... def mine(full_size, dataset, header, difficulty): target = zpad(encode_int(2**256 // difficulty), 64)[::-1] from random import randint nonce = randint(0, 2**64) while hashimoto_full(full_size, dataset, header, nonce) > target: nonce = (nonce + 1) % 2**64 return nonce difficulty x target = 2256
  • 16.
    16 II. Consensus AlgorithmPoW- Finality Problem Solve Bm Solve Bn Bm Two different miners may generate a block simultaneously. Bm Bn Bn
  • 17.
    17 II. Consensus AlgorithmPoW- Finality Problem Solve Bm2 Solve Bn2 Bm Another two different miners may generate a block simultaneously again. Bm Bn Bn Bn2 Bn2Bm2 Bm2
  • 18.
    18 II. Consensus AlgorithmPoW- Finality Problem Bm Bn Bm Bn Bm Bn Bm Bn Bn       txb(b a, 10) txb(b a, 10)
  • 19.
    19 II. Consensus AlgorithmCasperFFG - Finality System  "Casper FFG" paper by Vitalik and Virgil PoS based Finality System which Overlays an existing PoW blockchain.  Accountability : Penalty  Dynamic validators  Modular overlay : a partial consensus mechanism
  • 20.
    20 II. Consensus AlgorithmCasperFFG - Process 1) Voting  Vote for link(edge)  2 Commandments (Slashing conditions) 2) Justifying   2/3 votes 3) Finalizing  Canonical chain 1) 2) 3)
  • 21.
    21 AppendixReferences  Consensus  Byzantinefault  Reaching Agreement in the Presence of Faults (1980, M. PEASE, R, SHOSTAK, AND L. LAMPORT)  The Byzantine Generals Problem (1982, LESLIE LAMPORT, ROBERT SHOSTAK, and MARSHALL PEASE)  Practical Byzantine Fault Tolerance (1999, Miguel Castro and Barbara Liskov)  pBFT— Understanding the Consensus Algorithm  Proof-of-Work, Explained  Hashcash  Ethash