ArcBlock's Technical Learning Series presents Tendermint: Consensus made easy.
Tendermint is an opensource software aimed to securely and consistently replicating an application on many machines. It acts as a consensus engine and provides a generic application interface called ABCI which makes it pretty easy to integrate itself into other blockchain technology, like Ethereum.
This talk will give a high-level introduction to Tendermint, its architecture, how consensus works and how application data are replicated. We will also give some scenario about how Tendermint could be used to solve various problems.
https://www.arcblock.io
2. What is Tendermint
• software to securely and consistently replicating an application on many
machines
• securely: works even if 1/3 or machines failed
• consistently: every non-faulty machine sees the same tx log and compute the same state
(deterministic)
• two parts:
• blockchain consensus engine: Tendermint Core, ensures that same tx are reordered on every
machine in same order
• generic application interface: ABCI, enables the tx to be processed in any programming language
2
3. Tendermint vs zookeep / etcd / consul
• untrust env vs trust env
• trust env cannot tolerant a single Byzantine fault
• BFT vs paxos / raft
• can survive < 1/3 failure vs < 1/2 failure
• state machine replication vs kv store on top of classical consensus algo
3
4. Tendermint vs bitcoin / ethereum
• PoS vs PoW
• focus on hosting arbitrary application states in a p2p BFT env
• can be used as PnP replacement for consensus engine
• other blockchain codebase could be run as an ABCI application using tendermint
consensus
• e.g. ethermint, cosmos
4
5. Tendermint vs fabric / burrow
• fabric: similar idea on consensus
• burrow: EVM + tendermint + name registry / permission / …
5
7. Components in a blockchain app
• networking: for p2p connectivity and data replication
• mempool: for broadcasting tx
• consensus: for agreeing on most recent block
• storage: account states
• VM: for executing turning-complete contracts
• app logic: e.g. permissions
7
9. What is ABCI?
• An interface that connects Tendermint consensus engine with application
• Socket based protocol
• types are defined in protobuf: ABCI
9
11. ABCI interface: Block header
message Header {
// basic block info
string chain_id = 1 [(gogoproto.customname)="ChainID"];
int64 height = 2;
google.protobuf.Timestamp time = 3 [(gogoproto.nullable)=false, (gogoproto.stdtime)=true];
int64 num_txs = 4;
int64 total_txs = 5;
// prev block info
BlockID last_block_id = 6 [(gogoproto.nullable)=false];
// hashes of block data
bytes last_commit_hash = 7; // commit from validators from the last block
bytes data_hash = 8; // transactions
// hashes from the app output from the prev block
bytes validators_hash = 9; // validators for the current block
bytes next_validators_hash = 10; // validators for the next block
bytes consensus_hash = 11; // consensus params for current block
bytes app_hash = 12; // state after txs from the previous block
b t l t lt h h 13 // t h h f ll lt f th t f th i bl k
11
12. ABCI: Core APIs
• DeliverTx
• Each tx is delivered with this message
• application need to validate the tx received from DeliverTx against current state
• application then update the app state based on the tx
• CheckTx
• same as DeliverTx, but only do tx validation
• used by Tendermint Core
• mempool check the validity of tx with CheckTx
• relay valid tx to its peers
• Commit
• compute a cryptographic commitment to the current application state
12
13. Being deterministic
• without deterministic on tx processing on the p2p network, no consensus could be
formed.
• Things to consider:
• random number generators (without deterministic seeding)
• race conditions on threads (or avoiding threads altogether)
• the system clock (you can’t rely on it, e.g. the seed)
• uninitialized memory (in unsafe programming languages like C or C++)
• floating point arithmetic: floating point determinism
• same compiler
• same platform
• same CPU instruction set
• language features that are random (e.g. iteration on map)
13
16. Concepts
• validators: the node who participants in the protocol, take turns proposing blocks
of transactions and voting on them
• height: the number of the block
• round: a turn that a validator gets to propose a block for the height
• propose: a validator generate a block for the height and broadcast it out for
validating
• prevote: if the proposed block is valid, the validator prevote the block and
broadcast prevote message out and also listen and accumulate the prevotes from
others
• precommit: if 2/3 of validators prevoted the block, the validator precommit the
block and broadcast precommit message out and also listen and accumulate the
precommits from others
• commit: if 2/3 of validators precommitted the block, the block is in committed state
16
17. Stake
• voting power: not all validators will have the same “weight” in the consensus protocol
• Proof-of-Stake: voting power is denominated in a native currency
17
25. Genesis
• genesis_time: official time of blockchain start
• chain_id: unique id for the blockchain. Shall be less than 50 symbols
• validators: list of initial validators. Could be override by app
• pub_key: validator’s pubkey
• power: voting power
• app_hash: expected application hash (returned by ResponseInfo ABCI message), upon
genesis if app’s hash does not match, Tendermint will panic.
• app_state: application state, e.g. initial distribution of tokens
25
27. Start tendermint
• cmd: tendermint node
• by default it connect to ABCI app on 127.0.0.1:26658
• for unix domain socket: do: `tendermint node –proxy_app/var/run/your-app.sock
• in-process app: counter , kvstore , nil
• apps provided by ex_abci: counter, simple_chain
27
28. Start tendermint example
$ tendermint node --proxy_app=kvstore // the in-process app
I[10-16|16:01:06.946] Starting multiAppConn module=proxy impl=multiAppConn
I[10-16|16:01:06.946] Starting localClient module=abci-client connection=query impl=localClient
I[10-16|16:01:06.946] Starting localClient module=abci-client connection=mempool impl=localClient
I[10-16|16:01:06.946] Starting localClient module=abci-client connection=consensus impl=localClient
I[10-16|16:01:06.946] ABCI Handshake module=consensus appHeight=0 appHash=
I[10-16|16:01:06.946] ABCI Replay Blocks module=consensus appHeight=0 storeHeight=0 stateHeight=0
I[10-16|16:01:06.947] Completed ABCI Handshake - Tendermint and App are synced module=consensus appHeight=0 appHash=
I[10-16|16:01:06.947] This node is a validator module=consensus addr=CE925800DC1F3AF72721343A0FE67F1829
I[10-16|16:01:06.952] P2P Node ID module=p2p ID=e7f354ef522497ba97722bf6658b4e06b1f390ab f
I[10-16|16:01:06.953] Starting Node module=main impl=Node
I[10-16|16:01:06.953] Starting EventBus module=events impl=EventBus
I[10-16|16:01:06.953] Starting RPC HTTP server on tcp://0.0.0.0:26657 module=rpc-server
I[10-16|16:01:06.953] Starting P2P Switch module=p2p impl="P2P Switch"
I[10-16|16:01:06.953] Starting MempoolReactor module=mempool impl=MempoolReactor
I[10-16|16:01:06.953] Starting BlockchainReactor module=blockchain impl=BlockchainReactor
I[10-16|16:01:06.953] Starting ConsensusReactor module=consensus impl=ConsensusReactor
I[10-16|16:01:06.953] ConsensusReactor module=consensus fastSync=false
I[10-16|16:01:06.953] Starting ConsensusState module=consensus impl=ConsensusState
I[10-16|16:01:06.953] Starting baseWAL module=consensus wal=/Users/tchen/.tendermint/data/cs.wa
I[10 16|16 01 06 954] St ti Ti tTi k d l i l Ti tTi k
28
29. What is WAL?
• Write-Ahead Logging
• Usage:
• to ensure data integration, usually used in database
• changes to data files must be written only after those changes has been logged (and flushed to
permanent storage)
• Pros:
• reduce disk writes significantly - we don’t need to flush data pages to disks every time something
changes
• easy and efficient to sync
• any changes to the data can be redone from the WAL
29
30. Tendermint consensus
I[10-16|16:01:07.954] Timed out module=consensus dur=997.249ms height=1 round=0 step=Round
I[10-16|16:01:07.955] enterNewRound(1/0). Current: 1/0/RoundStepNewHeight module=consensus height=1 round=0
I[10-16|16:01:07.955] enterPropose(1/0). Current: 1/0/RoundStepNewRound module=consensus height=1 round=0
I[10-16|16:01:07.955] enterPropose: Our turn to propose module=consensus height=1 round=0 proposer=CE925800DC1F3
I[10-16|16:01:07.962] Signed proposal module=consensus height=1 round=0 proposal="Proposal{1/0
I[10-16|16:01:07.963] Received proposal module=consensus proposal="Proposal{1/0 1:1A00E597FAD3
I[10-16|16:01:07.964] Received complete proposal block module=consensus height=1 hash=C896B7C7ACBB819C762F29BF
I[10-16|16:01:07.964] enterPrevote(1/0). Current: 1/0/RoundStepPropose module=consensus
I[10-16|16:01:07.964] enterPrevote: ProposalBlock is valid module=consensus height=1 round=0
I[10-16|16:01:07.965] Signed and pushed vote module=consensus height=1 round=0 vote="Vote{0:CE925800D
I[10-16|16:01:07.965] Added to prevote module=consensus vote="Vote{0:CE925800DC1F 1/00/1(Prevot
I[10-16|16:01:07.965] enterPrecommit(1/0). Current: 1/0/RoundStepPrevote module=consensus height=1 round=0
I[10-16|16:01:07.966] enterPrecommit: +2/3 prevoted proposal block. Locking module=consensus height=1 round=0 hash=C896B7C7
I[10-16|16:01:07.966] Signed and pushed vote module=consensus height=1 round=0 vote="Vote{0:CE925800D
I[10-16|16:01:07.967] Added to precommit module=consensus vote="Vote{0:CE925800DC1F 1/00/2(Precom
I[10-16|16:01:07.967] enterCommit(1/0). Current: 1/0/RoundStepPrecommit module=consensus height=1 commitRound=0
I[10-16|16:01:07.968] Commit is for locked block. Set ProposalBlock=LockedBlock module=consensus height=1 commitRound=0 blo
I[10-16|16:01:07.968] Finalizing commit of block with 0 txs module=consensus height=1 hash=C896B7C7ACBB819C762F29BF
30
41. Requirement
• just replicate ethereum’s MPT based account system
• A wallet consists of a key pair that user can send tokens
• Tendermint as the consensus layer
• no EVM to execute smart contracts
• the whole app state is changed tx after tx, and the app state of the last tx of a block is
committed back to tendermint as app root
41
42. Implementation
• ex_abci
• gen_server for handling: info, check_tx, begin_block, deliver_tx, end_block, commit,
flush
• merkle_patricia_tree to handle MPT (using LevelDB)
• keccakf1600_orig for sha3
• libsecp256k1 for signing and verifying
42
44. MPT
• Purpose: Maintain the Merkle Patricia Tree
• Interfaces:
• open: open a level db and restore last committed root_hash, if any
• put: update a state for a key (account address) to the trie
• get: get a state from a key from the trie
• update_block: update last_block height, and put the root_hash for the height
• get_app_hash: get the root_hash for the last block
• get_info: retrieve last_block, last_block_hash
44
45. Account
• Purpose: Maintain account system
• Interfaces:
• get: get the state of the account in the world state
• put: update the state to the account in the world state
45
46. Tx
• Purpose: sign and verify transactions
• Interfaces:
• sign: calculate the sha3 of the tx and then do ecdsa sign
• verify: calculate the sha3 of the tx and then verify against the signature
46
47. Wallet
• Purpose: create wallet, create tx and retrieve the state from the chain, acting as a
client library
• Interfaces:
• new: create a new wallet with key pair and wallet address, using sha3 and ecdsa
• declare: a special transaction that could get 10000 genesis tokens. For test purpose.
• transfer: move tokens between accounts, tx will be signed by sender. Validator will verify the
signature and make sure these conditions are satisfied:
• from / to are not the same address
• nonce is expected
• balance is greater or equal to the total tokens in the tx
• balance: get balance of an address
• nonce: get nonce of an address
• info: get info of an address
• chain_info: get the status of the chain
47