Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Blockchain Scalability - Architectures and Algorithms


Published on

My presentation on 'Blockchain Scalability - Architectures and Algorithms' for the TechAthena Digital Community Webinar.

Blockchain Scalability is one of the most significant concern for Minimum Viable Blockchain Implementation. It is one of the key aspects determining the relevance and feasibility of Blockchain Technology for a particular use case.This session will cover the fundamental aspects of distributed computing that determine the contours of scalability.

Subsequently, the session will outline the parameters and metrics related to Blockchain Scalability in detail. In this context, the session will deep dive into architectural and algorithmic techniques that enables a scalable Blockchain.Architectural techniques such as vertical scaling and horizontal scaling will be explained in detail. Design techniques such as State Channels, Sharding, SideChains, Off chain computations, Block Size and Time Optimization etc. will be explained.

In summary, this session will conclude with the implications and trade-off between Blockchain Scalability, Security, Simplicity and Interoperability. Looking forward to your views and thoughts !

Published in: Software

Blockchain Scalability - Architectures and Algorithms

  1. 1. Gokul Alex, Associate Director, PwC India Blockchain Scalability Algorithms and Architectures
  2. 2. What is the scalability challenge in Blockchain ?
  3. 3. –Vitalik Buterin “Currently, all blockchain consensus protocols that are actively in use have an important limitation: every fully participating node in the network must process every transaction. ”
  4. 4. Interpretations of the Scalability Challenge • This gives the blockchain a high amount of security because of how much validation goes into each block • At the same time it means that an entire blockchain is only as fast as its individual nodes and not the sum of their parts.
  5. 5. Blockchain Scalability - Architectures and Algorithms Blockchain Trilemma A Blockchain can have at most two of these three properties : • Decentralisation • Scalability • Security
  6. 6. Blockchain Trilemma : Implications
  7. 7. Decentralisation, Transparency and Scalability has become new concepts of value
  8. 8. How do you measure the performance of distributed systems ?
  9. 9. Speedup, Efficiency & Scalability
  10. 10. Blockchain Scalability - Architectures and Algorithms Understanding SpeedUp, Efficiency and Scalability in Totality • Speedup measures how the rate of doing work increases with the number of processors • Efficiency E measures the work rate per processor • Scalability ψ(k1 ,k2 ) from one scale k1 to another scale k2 is the ratio of the efficiency figures
  11. 11. Blockchain Scalability - Architectures and Algorithms Understanding Scalability The number of transactions per unity time that the system can process
  12. 12. Scale Cube from Art of Scalability
  13. 13. Scale Cube and the contours of scalability
  14. 14. Performance Considerations from Scale Cube
  15. 15. Scale Cube and the Service Design Perspective
  16. 16. Blockchain Scalability - Architectures and Algorithms Decentralised Scaling : Cost and Constraints It should be noted that scaling does not imply a decrease in the latency of transactions. Indeed, improvements to scaling may sometimes come at the cost of increased latency.
  17. 17. Blockchain Scalability - Architectures and Algorithms Scalability and Consensus : Myths and Misconceptions It should also be noted that, on their own, consensus mechanisms do not affect transaction throughput in any meaningful way. A common misconception is that “Proof-of- Work (PoW) is slow” — that could not be further from the truth.
  18. 18. Blockchain Scalability - Architectures and Algorithms Strategic approach to Scalability across Consensus Systems that achieve greater scalability with different consensus mechanisms do so almost universally by reducing the size of the set of block producers. If the root cause of the scalability problem is that every transaction must be validated by every node, one way forward is to simply reduce the number of nodes on the network
  19. 19. Blockchain Scalability - Architectures and Algorithms Understanding Decentralisation A system is decentralised if and only if it is: • Distributed • Trustless • Permissionless
  20. 20. Blockchain Scalability - Architectures and Algorithms Blockchain Scalability Factors and Constraints Size of transactions Size of a block How many transactions in a block How often blocks get added to the chain How nodes collaborate in the chain How nodes add transactions to the chain
  21. 21. Blockchain Scalability - Architectures and Algorithms Blockchain Scale in comparison with Visa Bitcoin - 90,000 Transactions / Day Visa - 150 Million Transactions / Day
  22. 22. Blockchain Scalability - Architectures and Algorithms Common approaches for Blockchain Scalability Off chain computations Side Chains State Channels Sharding Protocols New Consensus Protocols Reducing the Block size
  23. 23. Blockchain Scalability - Architectures and Algorithms Layer 1 - Layer 2 Scaling approaches The class of proposals to properly scale decentralised blockchains currently revolve around not having each node in the network validate every transaction. This is accomplished by either sharding the base blockchain (so-called layer-1 scaling) or having separate chains running alongside the base chain that only a subset of users need to fully validate (so-called layer-2 scaling).
  24. 24. Blockchain Scalability - Architectures and Algorithms Scalability Measures • Maximum throughput - maximum rate at which the blockchain can confirm transactions • Latency - time for a transaction to be confirmed. The transactions are confirmed once they are included in a block, which is roughly 10 minutes for bitcoin. • Bootstrap time - The time it takes for a new computer node to download the history necessary to validate a new transaction
  25. 25. Blockchain Scalability - Architectures and Algorithms Scalability - Hard Truths In all blockchain protocols, each node stores all states with account balances, contract code, and entire transaction history, in order to verify a transaction’s validity for each transaction in the network without a trusted third party. This provides a large amount of security but limits scalability to not being able to process more transactions than any single node in the network can.
  26. 26. Blockchain Scalability - Architectures and Algorithms Scalability Dimensions Node Identity Management Consensus Finality Node Scalability Client Scalability Performance throughput Performance latency Power consumption Tolerated power of an adversary Network synchrony Correctness proofs
  27. 27. Blockchain Scalability - Architectures and Algorithms Node identity management How nodes are identified in a consensus protocol in a network through a unique identifier In a proof of work based network, anyone can participate and contribute to the system In a BFT consensus, every participating node need to identify to participate in the transactions This results in a centralised identity management requirement in a BFT protocol implementation
  28. 28. Blockchain Scalability - Architectures and Algorithms Consensus Finality • Consensus Finality is the property that ensures that a valid block added to the blockchain is not removed • PoW violates this property when two nodes append a block to the same block which result in a fork • In the case of Bitcoin the forks are handled by the longest chain rule, thus breaking the consensus finality by removing the shorter chain or the GHOST rule • BFT satisfies consensus finality with immediate confirmation
  29. 29. Blockchain Scalability - Architectures and Algorithms Client Scalability When it comes to scalability with the number of clients, both PoW and BFT support thousands of clients all connected at once with good concurrency and parallelism
  30. 30. Blockchain Scalability - Architectures and Algorithms Node Scalability Scalability in Proof of Work Scalability in BFT Scalability in DAG Scalability in PoS Scalability in DPoS Scalability in DPoW
  31. 31. Blockchain Scalability - Architectures and Algorithms Performance Issues in PoW PoW scalability is reliant on the block size and the rate of the block creations If the block size is increased, potential trees are created in the blockchain leading to double spend attacks
  32. 32. Blockchain Scalability - Architectures and Algorithms Early Proposals for Bitcoin Scalability Removing old transactions from Blockchain and a database is used to hold the non-empty address trees In this way, nodes that are validating the transactions do not have to store the previous transactions that are not relevant to them Bitcoin Next Generation is another proposal where the blocks are decoupled and there are leader blocks and micro blocks that handle transactions Miners would complete for the leader block, these would be the ones in charge of generation of new micro blocks
  33. 33. Blockchain Scalability - Architectures and Algorithms Double Spend attacks on Blockchain Ledger Double spend attacks is a method to override the main chain to reverse transactions The attacker pays a person and in secrecy builds a chain of blocks where the payment is not included. By releasing the chain the attacker can cause a replacement in the ledger where the payments are erased or redirects the payment to somewhere else
  34. 34. Blockchain Scalability - Architectures and Algorithms Double Spend attack on Bitcoin Network This requires a lot of computational power which makes the attack unlikely since the honest nodes have a lot of computational power, however there was a case of a mining pool in Bitcoin having over 40% of the total computational power. If the attacker has a lot of computational power there is a possibility that the attacker can generate blocks that could replace the honest longest chain and that enables the attacker to replace the main chain at will. When the attacker has more computational power than the honest nodes, it is called the 51% attack (also known as the majority attack)
  35. 35. Blockchain Scalability - Architectures and Algorithms Double Spend attack and Bitcoin Scalability aspects The Bitcoin protocol becomes more susceptible to double-spend attacks when it scales upwards and tries to meet the demand. If we assume that the attacker creates blocks at a rate that is faster than the rate of the honest main chain, the attacker will always be successful with these types of attacks regardless of the length of the chain it aims to replace .
  36. 36. Blockchain Scalability - Architectures and Algorithms State Channels approach to Blockchain Scalability State channels are the general form of payment channels, applying the same idea to any kind of state- altering operation normally performed on a blockchain.
  37. 37. State Channel Scalability Model
  38. 38. Blockchain Scalability - Architectures and Algorithms Prominent State Channel Implementations • Ligtning Network • Raiden Network • Trinity • Spankchain • Perun • Counterfactual • Celer Network • Machinomy • Fun Fair • Liquidity • Connext Network
  39. 39. Blockchain Scalability - Architectures and Algorithms Bitcoin Scalability and Lightning Network The Lightning Network is supported by numerous smart contracts put together in a system built on the topmost tier of the Bitcoin Blockchain. The protocol allows very fast transactions speeds that are accompanied by very low transitioning fees.
  40. 40. Blockchain Scalability - Architectures and Algorithms Building Blocks of Lightning Network Unconfirmed Transactions Double Spend Protection Multi signature Addresses Time Locks Hash values and Secrets
  41. 41. Lightning Network Lifecycle • Set up a wallet with a multi-signature feature with some amount in BTC • Upload the Wallet’s address into the public Bitcoin Blockchain. • This is accompanied by a smart contract that clearly states what amount of BTC belongs to whom. • Once the pay channel is instantiated, it opens up an avenue for the parties therein to undertake unlimited transactions amongst themselves. • The information in the wallet set is not updated onto the main Blockchain. The transactions occur off-chain. • Upon completion of every transaction, a balance is signed up by both parties, and this is reflected on the balance sheet. • At any given time, the multi-signature wallet will show the balances owed to each party. • In case of a dispute or should the payment channel be locked, the contractual obligations terminate there and the involved parties pay each other as per the balances reflected as a share in the Multi-signature wallet.
  42. 42. Lightning Network Simplified
  43. 43. Blockchain Scalability - Architectures and Algorithms Unconfirmed Transactions The Lightning Network is built up from more or less regular Bitcoin transactions. These transactions are typically not actually broadcast over the Bitcoin network. Instead, they are stored locally, on the nodes of users - but they can be broadcast over the network at any time.
  44. 44. Blockchain Scalability - Architectures and Algorithms Double Spend Protection If two transactions (or: inputs) rely on the same output, only one can confirm. Even unconfirmed transactions can be conflicting, meaning only one can ever confirm.
  45. 45. Blockchain Scalability - Architectures and Algorithms MultiSig ( P2SH) Addresses Multisig addresses are Bitcoin addresses that require multiple private keys to “unlock” and spend bitcoins from. The Lightning Network often uses two out of two (2-of-2) multisig set-ups. Unlocking bitcoins from 2-of-2 multisig addresses requires two signatures, from two dedicated keys.
  46. 46. Blockchain Scalability - Architectures and Algorithms Time Locks Time-locks can “lock bitcoins up” in an output, to make them spendable (to be included in a subsequent input) only at some point in the future. There are two different types of time-locks: the absolute type, called CheckLockTimeVerify (CLTV), and the relative type, CheckSequenceVerify (CSV). CLTV locks bitcoins up until a (more or less) concrete time in the future: an actual time and date, or a specific block height. CSV, instead, uses relative time. Once a CVS-output is recorded on the blockchain, it takes a specific amount of blocks from that point on before the bitcoins can be spent again.
  47. 47. Blockchain Scalability - Architectures and Algorithms Hash Values and Secrets In a bitcoin transactions, a hash can be included in an output, and require the subsequent input to include the corresponding value in order to be spendable.
  48. 48. Lightning Network - Security and Scale
  49. 49. Lighting Network - Transaction Model
  50. 50. Blockchain Scalability - Architectures and Algorithms Sidechain approach to Blockchain Scalability A sidechain is a separate blockchain that is attached to its parent blockchain using a two-way peg. In other words, you can move assets to the sidechain and then back to the parent chain.
  51. 51. Sidechain Scalability Model
  52. 52. Blockchain Scalability - Architectures and Algorithms Sidechain Implementations • Ethereum Plasma • Rootstock • Alpha • Liquid • Loom • POA Network • Bitcoin Extended • Hivemind • MimbleWimble • Elements Project • Bitcoin Codex
  53. 53. Blockchain Scalability - Architectures and Algorithms Ethereum Plasma approach to Blockchain Scalability Plasma is a series of contracts that run on top of a root chain (Ethereum main chain) and consists of a network of “child chains” connected to a root chain in a hierarchical, tree-like structure.
  54. 54. Ethereum Plasma Lifecycle • Initially, smart-contracts are created on the Ethereum main-chain. These smart contracts serve as the “root” of the Plasma child-chain. • This main chain entry contains the basic rules of the child- chain, records state hashes of the child-chain, and allows users to move assets between the Ethereum main-chain and the child-chain. • After rooting the child-chain in the main chain, a child- chain is created. This child-chain features its own consensus algorithm, independently from the Ethereum main-chain. • Once the child-chain is up and running, the block creators periodically commit a validation to the main-chain, essentially proofing that the current state of the child- chain is valid according to the consensus rules.
  55. 55. Plasma Consensus Protocol • The consensus mechanism for this proof of stake system, is again, enforced in an on- blockchain smart contract. • Instead of enforcement of an incrementing nonce state (via revocations), a system of fraud proofs is enforced for balances and state transitions of these chain hierarchies. • In effect, we are able to create state transitions which are only periodically committed to parent chains (which then flows to the root blockchain). • Constructs computation in a MapReduce format to more easily design computation and state transitions in a hierarchical tree. • This creates economically enforceable computation at scale, with only one block header/hash committed on the root chain to encompass very high amount of data and work. • It is only if a block is faulty that proof of invalidity is published, otherwise extremely minimal amounts of data is submitted on the root chain periodically.
  56. 56. Plasma Contract Interactions
  57. 57. Plasma Consensus Interactions
  58. 58. Plasma Transaction Dynamics
  59. 59. Blockchain Scalability - Architectures and Algorithms Ethereum Sharding approach to Blockchain Scalability Sharding is actually much older than blockchain technology and has been implemented in a variety of systems from business database optimizations to Google’s global Spanner database. Essentially, sharding is a particular method for horizontally partitioning data within a database. More generally, the database is broken into little pieces called “shards”, that when aggregated together form the original database.
  60. 60. Ethereum Sharding Model
  61. 61. Sharding Data Structure Sharding is often referred to as a Layer 1 scaling solution because it is implemented at the base-level protocol of ethereum itself.. Ethereum breaks down the network into specific shards. Each shard is assigned a specific group of transactions that is determined by grouping specific accounts (including smart contracts) into a shard. Each transaction group has the following structure. • Header • The shard ID of the transaction group • Assignment of validators through random sampling • State Root (state of the merkle root of the shard before and after transactions) • Body • All of the transactions that belong to the transaction group that are part of the specific shard.
  62. 62. Sharding Data Structure Diagram
  63. 63. Sharding and Transactions • Transactions are specific to each shard and occur between accounts native to that shard. • When transactions are verified, the state of the network changes and account balances, storage, etc are updated. • In order for the transaction group to verify as valid, the pre- state root of the transaction group must match the shard root in the global state. • If they match, the transaction group is validated and the global state is updated through the particular shard ID state root. • Instead of only containing a state root, each block of the Ethereum blockchain now contains both a state root and the transaction group root. • Basically, there is a merkle root of all of the different shards that contain the updated and verified transaction groups. This root is stored in the blockchain along with the updated state root.
  64. 64. Ethereum Sharding Model
  65. 65. Cross Shard Communication • The cross-shard communication is achieved through applying the concept of transaction receipts. • The receipt for a transaction is stored in a merkle root that can be easily verified but that is not part of the state root. • The shard receiving a transaction from another shard checks the merkle root to ensure that the receipt has not been spent. • Essentially, the receipts are stored in a shared memory that can be verified by other shards, but not altered. • Therefore, through a distributed storage of receipts, shards are able to communicate with each other.
  66. 66. Sharding and 1% attack • A major problem is the idea of a Single-Shard Takeover Attack, where an attacker takes over the majority of collators in a single shard to create a malicious shard that can submit invalid collations. • In a 100 shard system, it takes only 1% of network hash rate to dominate the shard • As a solution, random sampling of validators in each shard is recommended • Every shard will get assigned a bunch of validators and the ones that will actually be validating will be randomly sampled from that set.
  67. 67. Sharding and Validator Manager Contract
  68. 68. Blockchain Scalability - Architectures and Algorithms Blockchain Architecture Constraints and Challenges In a Blockchain, every fully participating node in a network must process every transaction Every blockchain protocol that works in this way is forced to choose between either limiting itself to a low maximum transaction throughput, with a resulting high per- transaction cost, or allowing a high degree of centralisation.
  69. 69. Blockchain Scalability - Architectures and Algorithms Understanding Constraints Physical resource constraints Software constraints Data communication is limited by the speed of light, bandwidth transmission limits, CPU processing capacity, network consistency, network availability, network partition tolerance Network latency is at the very base of the constraints It measures how long it takes for data to travel
  70. 70. Blockchain Scalability - Architectures and Algorithms Block Size Computation in Bitcoin Network At this theoretical rate of transactions that means the nodes in the network must be able to process 500 bytes x 2,000 tps = 1 MB amount of transactions per second. Processing a transaction involves hashing and ECDSA signature verifications. RIPEMD-160 and SHA256 (both hash algorithms) run at about 100 megabytes per second, so 2,000 transactions could be processed in about 10 milliseconds, so fast enough that we don’t need to worry about it.
  71. 71. Blockchain Scalability - Architectures and Algorithms Block Gas Limit Computation in Ethereum Ethereum is a little different in that it doesn’t have a traditional block size but a per block gas limit. Gas is the payment unit used to pay for the computations required to process the transaction. This is a consistently reevaluated value based on the current processing, storage and bandwidth conditions of the network. This is important because not every transaction is just used for transferring an asset but may be used to signal a contract or carry data to it which requires the network to do some level of extra computation.
  72. 72. Blockchain Scalability - Architectures and Algorithms Challenges of increasing Blockchain Transaction Rate If the rate of transactions increases by way of block size increase or block generation rate, the blockchain would grow at a corresponding rate. Currently, the Bitcoin blockchain grows at a max rate of about 1 GB a week (max rate of about 4 GB for SegWit enabled Bitcoin). If the block size is doubled, that rate would be 2 GB a week, tripled it would be 3 GB a week and so on.
  73. 73. If the Blockchain scalability demands larger computational machines, it will result in centralisation of network among the machines with better computation power and capacity
  74. 74. Blockchain Scalability - Architectures and Algorithms Determinants of Blockchain Throughput The throughput of the blockchain can be affected by two elements, the size of the block and the block creation rate. The difficulty of the computational problem to create a valid block can also be lowered so that the creation of blocks is accelerated. Increasing the block size or increasing the rate of creation of blocks influence the blockchain in a negative way by introducing more forks which in turn reduces the security threshold of the blockchain.
  75. 75. Blockchain Scalability - Architectures and Algorithms Block size incrementation and Blockchain Scalability BIP 100 is about changing the 1 MB block size to a new floating block size which is determined by the consensus mechanism of the miners. Another suggestion is to increase the block size by 4.4% each 97 days until the year 2063 which is a 17.7% increase per year.
  76. 76. Blockchain Scalability - Architectures and Algorithms Segregated Witness SegWit is a protocol to increase the block capacity and to provide protection from transaction malleability Segregated witness approach does not increase the block size limit, but it increases the amount of transactions that can be stored in a block. This approach in the best case scenario increases the throughput by four times. The segregated witness approach resolves the transaction malleability problem and thus allows new mechanisms to be implemented which could provide powerful tools for the scalability issues in blockchain implementations
  77. 77. Blockchain Scalability - Architectures and Algorithms The Greedy Heaviest Observed Sub Tree The basic concept behind the protocol is that blocks that are off the main chain can still contribute to its weight It offers performance benefits over the standard longest chain protocol as is implemented in Bitcoin This allows the network to set higher rates for block creation and increase the block size without worrying about the 51% attacks which means a higher transaction throughput can occur within the network
  78. 78. Blockchain Scalability - Architectures and Algorithms What happens if you split the data into sub chains It has the two fundamental problems it decreases security, to the point where the security of each individual blockchain on average does not grow at all as total usage of the ecosystem increases, It massively reduces the potential for interoperability, as applications no longer have the ability to talk to each other with zero latency and total security over a single ledger Alternatively we can use cumbersome ”atomic cross-chain” interaction protocols
  79. 79. Blockchain Scalability - Architectures and Algorithms Merge Mining Proof of work miners mining multiple chains simultaneously which could best offer a linear trade off between a thousand chains and a single chain extreme If each chain is mined by only one in a thousand miners, then it is vulnerable to very cheap attacks If each chain is mined by every miner then the consensus participant must process every transaction anyway
  80. 80. Blockchain Scalability - Architectures and Algorithms Checkpointing Techniques involving “checkpointing” a smaller blockchain inside a larger one also fail to provide security, as they do not prevent an attacker with majority participation in the smaller blockchain’s consensus process from creating a checkpoint of fake or unavailable data.
  81. 81. Blockchain Scalability - Architectures and Algorithms Scalability and Implicit Validation Processes The problem of simultaneously achieving the best of both worlds: having only a small portion of consensus participants explicitly participate in validating each transaction, but have the entire weight of the network implicitly stand behind each one, has proven surprisingly hard.
  82. 82. Blockchain Scalability - Architectures and Algorithms Light Clients and the likelihood of scalable validations Nodes must necessarily employ statistical and economic means in order to make sure that other blocks are secure In essence, every node can only fully keep track of at most a few parts of the entire “state”, and must therefore be a “light client”, downloading only a small amount of header data and not performing full verification checks,
  83. 83. Blockchain Scalability - Architectures and Algorithms Data Availability issues and impact on Blockchain Scalability It is entirely possible for a block to appear that looks valid, and is valid, but for which the auxiliary data is unavailable, leading to a situation where no other validator can effectively validate transactions or produce new blocks since the necessary data is unavailable to them.
  84. 84. Blockchain Scalability - Architectures and Algorithms Parallel processing and State Transition Functions When transactions are processed by different nodes in parallel, it is not possible to parallelise arbitrary computation quite easily We must determine the set of restrictions to a state transition function that will best balance parallelisability and utility.
  85. 85. Blockchain Scalability : Need for Convergence !