Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Hyperledger Indy Public Blockchain Node

469 views

Published on

https://ssimeetup.org/hyperledger-indy-public-blockchain-node-alexander-shcherbakov-webinar-43/
Alexander Shcherbakov is a software engineer at DSR working on the team at Evernym. He has a Ph.D. in Mathematics and is one of the maintainers of Hyperledger Indy and Hyperledger Plenum. In this presentation, he will explain the value of a decentralized ledger in an SSI ecosystem and examine Hyperledger Indy, which is the distributed ledger that has been powering the Sovrin Network for more than two years.

Our identities have to be trusted to be useful. When we meet strangers, we decide how much we trust them by what they tell us, and whether a trusted third party will vouch for them. In traditional identity systems, the trusted third party knows everything about everyone in the ecosystem. In Self-Sovereign Identity systems, we rely on a decentralized ledger to privately validate that the identity claims do in fact come from a trusted issuer.

Indy’s blockchain implementation is Plenum, which is a general purpose, public-permissioned, BFT distributed ledger. The presentation takes a technical look at the architecture, cryptography, transactions, data structures, and storage of the ledger including auditability, request processing, catch-up procedure, and support for custom plugins and custom transactions.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

The Hyperledger Indy Public Blockchain Node

  1. 1. Hyperledger Indy Public Blockchain Presented by Alexander Shcherbakov Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  2. 2. 1. Empower global SSI communities 2. Open to everyone interested in SSI 3. All content is shared with CC BY SA SSIMeetup.org Alex Preukschat @SSIMeetup @AlexPreukschat Coordinating Node SSIMeetup.org https://creativecommons.org/licenses/by-sa/4.0/ SSIMeetup objectives
  3. 3. ● Indy has its own implementation of Distributed Ledger not dependent on any other blockchain platform ● Indy has its own implementation of a PBFT-like consensus protocol Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  4. 4. ● Indy is one of active Hyperledger projects ● Indy deployment (Sovrin) is in production for more than 2 years Sovrin Networks: ● Builder Net ● Staging Net ● Main Net Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  5. 5. Agenda 1. SSI and Public Blockchain 2. Indy-Plenum and Indy-Node 3. Architecture Overview 4. Ledger 5. Consensus Protocol ○ RBFT ○ Moving to Aardvark ○ Plenum protocol specific 6. Summary and Key Features Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  6. 6. SSI and Public Blockchain: Verifiable Credentials ProverIssuer Verifier Issues Credential Presents Credential Decentralized Identifiers (DIDs) Signs Credential Countersigns Credential Verifies Signatures Wallet Zero Know-ledge Encoding Zero Know-ledge Proof Public Blockchain Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  7. 7. SSI and Public Blockchain: Why? ● Why Blockchain? ○ Decentralized source of trust for publicly available information ● Why Blockchain is Public? ○ Anyone in the ecosystem need to be able to read data (such as public keys, schemas, etc.) Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  8. 8. SSI and Public Blockchain: What data is on Blockchain ● No private data is written to the Blockchain ● Only Public data is there ● What is on Blockchain: ○ Public DIDs ○ Issuer’s Public Keys ○ Credential’s Schema ○ Information about revocation ● What is not on Blockchain: ○ Pairwise DIDs ○ Credentials ○ Proves ○ Private keys Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  9. 9. Indy-Plenum and Indy-Node ● Indy-Plenum: ○ https://github.com/hyperledger/indy-plenum ○ Consensus Protocol ○ Ledger ● Indy-Node: ○ https://github.com/hyperledger/indy-node ○ Depends on indy-plenum ○ Identity-specific transactions Indy-Plenum Indy-Node SCHEMA txn CRED_DEF txn GET_SCHEMA request GET_CRED_DEF request Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  10. 10. Indy-Plenum and Indy-Node ● Indy is a Ledger purpose-built for Identity ● Can be used as a general-purpose Ledger ○ Extend Plenum ○ Custom transactions (pluggable request handlers) ○ Plugins Plenum Plugin A Txn C Txn D Txn A Txn B Plugin B Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  11. 11. Indy-Plenum and Indy-Node ● Written in Python ● Depends on ○ ZMQ ○ Indy-crypto (Hyperledger Ursa) ○ Libsodium ● Message-driven and modular architecture ○ Recent refactorings improved this ● Extensive test coverage ○ TDD ○ Unit tests ○ Integration tests ○ Property-based and simulation tests ○ System tests ○ Load tests (usually 25 Nodes) Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  12. 12. Architecture Overview: Indy Blockchain Type Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  13. 13. Architecture Overview: Validator and Observer Nodes ● Validator ○ Handles Writes and Reads ○ These are the nodes that come to consensus ● Observer* ○ Handles Reads ○ Keep their “state” in sync with the Validators *Partially implemented Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  14. 14. Architecture Overview: Validator Nodes Plenum Consensus Protocol (RBFT) ● Each Node replicates all ledgers ● Each Ledger has a Merkle Tree ● Most of the Ledgers have State based on Patricia Merkle Trie ZMQ as secure transport ● TCP-based ● CurveCP, libsodium ● Authenticated encryption, no digital signatures ○ Authentication: Poly1305 MAC ○ Symmetric key crypto: XSalsa20 ○ Public Key crypto: Curve25519 N=3F+1 ● N - number of nodes ● F - max number of malicious nodes BLS multi-sig Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  15. 15. Architecture Overview: Write Requests ● (Multi) Signed by the user ● Digital Signature: Ed25519 F+1 equal replies Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  16. 16. Architecture Overview: Read Requests Just 1 Reply: ● BLS multi-sig ● State (audit) proof No signature Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  17. 17. Architecture Overview: Authentication Authentication is based on the information present in the Ledger ● Write Requests: ○ Must be signed (Ed25519 digital signature) ○ Signature is verified against a Public Key stored on the Ledger (NYM txn) ○ Every transaction author must have a DID and a Public Key (NYM txn) on the Domain Ledger ● Read Requests: ○ Anyone can read, no authentication is required Write Request Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  18. 18. Architecture Overview: Authorization Authorization is based on the information present in the Ledger ● Write Requests: ○ There is a role associated with every DID ○ There are configurable auth rules (stored in Config Ledger) which can define authorization policy for every action ○ The rules may define how many signatures of the given role are required ○ The rules can be composed by OR/AND expressions ● Read Requests: ○ Anyone can read, no authorization is required Add a new SCHEMA: (1 TRUSTEE) OR (2 STEWARDS) Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  19. 19. Ledger: Transaction Log and Merkle Tree ● Ledger: ○ Ordered log of transactions ○ Merkle Tree for the whole ledger ○ No real blocks ● RocksDB as key-value storage ● MessagePack for serialization ● Ledger catch-up procedure ○ On Start-up ○ On lagging behind Transaction Log Merkle Leaves Merkle Nodes Merkle Tree Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  20. 20. Ledger: Merkle Tree 1. Merkle Tree Root Hash ○ Ledger Catchup ○ Transaction Validation 2. Consistency Proof ○ Ledger Catchup 3. Inclusion (audit) Proof ○ Reply to written transaction ○ GET_TXN reply Audit Proof for d3 transaction Appended transactions Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  21. 21. Ledger: Ledger Types Indy has multiple Ledgers (each with a separate transaction log and a merkle tree): ● Audit Ledger ○ Order across ledgers ● Pool Ledger ○ Transaction for every Node in the pool ○ Adding, editing, removing nodes ● Config Ledger ○ Pool config parameters ○ Used in transaction validation ● Domain Ledger ○ Identity-specific transactions ○ Application-specific transactions ● Plugins can add new ledgers Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  22. 22. Ledger: Pool Ledger ● A new Pool is built from genesis transactions ● Nodes can be added and removed from the Pool by sending a NODE txn to the Pool Ledger ● Node’s data can be modified by sending a NODE txn to the Pool Ledger 1: add Node1 2: add Node 2 3: add Node 3 4: add Node 4 5: edit IP address for Node 1 6: add Node 5 7: add Node 6 8: remove Node 2 9: remove Node 3 Node 1 Node 4 Node 5 Node 6 Pool Ledger Genesis transactions Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  23. 23. Ledger: Audit Ledger ● Why ○ Synchronization between ledgers ■ Global sequence number between ledgers ■ Ledgers are caught up sequentially and one by one ○ Recovering of pool state after startup ○ External audit ● Audit transaction as a Block: ○ Batch seq no ○ View no ○ Corresponding ledger root hash ○ Corresponding ledger size ○ Current Primaries 1: N1 pool txns 2: N2 domain txns 3: N3 pool txns 4: N4 config txns 5: N5 domain txns 6: N6 domain txns ….. Pool ledger Domain ledger Config ledger Audit ledger Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  24. 24. State ● Each Ledger (except Audit Ledger) has a State ○ Pool State ○ Config State ○ Domain State ● Map ordered list of transactions to the current state as dictionary ○ Dynamic Validation ○ Read requests. ● Merkle Patricia Trie (as in Ethereum) ○ Radix Tree + Merkle Tree ○ Ledger Merkle Tree for Lists (ordered txn log) ○ Patricia Merkle Trie for Dicts ● Key-value storage - RocksDB. 10: DID_A, PUB_KEY_A1 … 24: DID_A, PUB_KEY_A2 … 36: DID_B, PUB_KEY_B1 … 102: DID_B, PUB_KEY_B2 … 125: DID_A, PUB_KEY_A3 {DID_A : PUB_KEY_A3, DID_B : PUB_KEY_B2} Ledger State Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  25. 25. Consensus Protocol: BFT ● No generals trust any other one general ● Each independently decides to attack, if two others also commit to attack ● With four generals, we can have one faulty general, and we can still agree Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  26. 26. Consensus Protocol: RBFT ● Byzantine Fault Tolerance ○ Built on RBFT: Redundant Byzantine Fault Tolerance. ○ Improves over PBFT (by Miguel Castro and Barbara Liskov) by executing several protocol instances in parallel ● Better throughput, lower latency than proof-of-work ● Performs better compared to its predecessors under dynamic load and under attack Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  27. 27. Consensus Protocol: RBFT Three Phase Commit Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  28. 28. Consensus Protocol: RBFT Redundancy with Active Monitoring Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  29. 29. Consensus Protocol: View Change ● Protocol is leader-based ● Leader may behave maliciously ○ Disconnected/Stopped ○ Degraded performance ○ Inconsistent Data (Ledger/State) ● If the Pool realizes that a Leader needs to be changed, it starts a View Change process ○ RBFT has multiple instances of the protocol that compare performance, and decide if master protocol is degraded ● View Change is implemented the same way as in original PBFT paper ○ A variant without digital signatures ● Plenum has a couple of enhancements to make sure the data is consistent during the View Change Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  30. 30. Consensus Protocol: View Change ● All transactions that could be potentially ordered on at least one correct Node are eventually ordered on all Nodes ● View Change procedure: ○ Each node propagates its prepared certificate to other nodes (that is transaction it could potentially ordered) ○ A new Leader decides which transactions need to be re-ordered and do the re-ordering Node 1 Node 2 Node 3 Node 4 Checkpoint: ppSeqNo=100 Prepared: ppSeqNo=120 Checkpoint: ppSeqNo=100 Prepared: ppSeqNo=120 Checkpoint: ppSeqNo=100 Prepared: ppSeqNo=119 Checkpoint: ppSeqNo=100 Prepared: ppSeqNo=115 Re-order from ppSeqNo=100 till ppSeqNo=120 Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  31. 31. Consensus Protocol: Moving to Aardvark ● Although RBFT protocol may be quite sensitive to malicious Leaders in some conditions, it’s slower than other PBFT-like protocols ○ N^3 vs N^2 ● We are expecting to change consensus protocol to Aardvark ○ PBFT-like protocol with the same view change implementation ○ Has just 1 protocol instance (like in PBFT and unlike RBFT) ○ Does regular View Changes ○ Probability of View Change depends on the Leader’s performance Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  32. 32. Plenum Protocol Specific ● 3PC Batching ○ Multiple transactions are ordered as one in a batch (block) ● Data Consistency check as part of Consensus Protocol ○ Apply batches as proposed by the Leader to the Ledgers and States => uncommitted merkle root ○ Compare uncommitted merkle root hash with the Leader’s one (in PrePrepare message) ○ This guarantees Data Consistency ○ If Leader sends inconsistent Data - View Change happens Txn1 Txn2 TxnN Applied batches Merkle root A PRE_PREPARE: - Batch of txns - Leader’s merkle root: root B Merkle root B 3PC Batch Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  33. 33. Plenum Protocol Specific ● Dynamic validation based on the current uncommitted state ○ When a PrePrepare is applied, each transaction must pass the dynamic validation ○ Dynamic validation is performed against the current uncommitted Ledger or State ● Usage of Audit Ledger ○ Audit Ledger is used to confirm data consistency as part of consensus ○ Audit Ledger’s root is used Checkpoint Txn 1 Txn 2 Txn 3 3PC Batch (PrePrepare) Txn 1 Txn 1 Txn 2 Verified against Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  34. 34. Plenum Protocol Specific ● Sequential applying of PrePrepares ○ We may have more than one Batch (PrePrepare) in flight, but all PrePrepares are applied sequentially (no gaps) to check data consistency ● Message Requests ○ If a message from a Node is lost/missing, it’s requested from this Node PrePrepare: ppSeqNo=10 PrePrepare: ppSeqNo=11 PrePrepare: ppSeqNo=13 PrePrepare: ppSeqNo=12 Apply Apply Do not Apply - Apply ppSeqNo=12 - Apply ppSeqNo=13
  35. 35. Plenum Protocol Specific: BLS multi-signature Sufficient to send Read requests to just one Node: ● State (Audit) Proof ○ Merkle Tree Proof that the result belongs to a State (Ledger) Merkle Tree with the given root ● BLS multi-signature against the merkle tree root ○ All nodes multi-sign the merkle tree root of Ledgers and States as part of Consensus Procedure The client verifies State (Audit) Proof and BLS multi-sig We trust the root as it was signed by the nodes in the pool Read Request Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  36. 36. Plenum Protocol Specific: BLS multi-signature ● BLS multi-signature as part of Consensus Protocol ○ Each Node BLS signs data during Consensus ■ Ledger merkle root hash ■ State merkle root hash ■ Timestamp ○ BLS multi-signature is calculated once the Batch is ordered ○ If there is no requests in the Pool, a PrePrepare with no requests is sent to update the BLS multi-signature Node 1 Node 2 Node 3 Node 4 BLS signature 2 in COMMIT BLS signature 4 in COMMIT BLS signature 3 in COMMIT BLS multi-signature from BLS signatures 2, 3, 4 Example of BLS multi-sig calculation for Node 1 The same is applied to every Replica Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  37. 37. Cryptography Summary ● Ledgers: ○ Merkle Tree (Ledger) ○ Patricia Merkle Trie (State) ● Node-to-Node Communication ○ ZMQ (libsodium) as secure transport ■ CurveCP handshake ■ Authenticated Encryption ● Authentication: Poly1305 MAC ● Symmetric key crypto: XSalsa20 ● Public Key Crypto: Curve25519 ■ No Digital Signatures ○ BLS multi-signature to sign merkle roots ● Client-to-Node communication ○ Ed25519 Digital Signatures Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  38. 38. Summary ● Ledger purpose-built for Identity ● Indy has its own Ledger and consensus protocol implementation ● Indy is in production (Sovrin network) for more than 2 years ● Indy Consensus Protocol: ○ RBFT consensus protocol with a potential plan to move to Aardvark ● Indy Ledger: ○ Multiple Ledgers (each with Merkle Tree) ○ States for efficient reads and validation ○ Authentication, Authorization and dynamic validation is based on the information from the Ledger ○ Audit Ledger synchronizes the ledgers and introduces blocks Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  39. 39. Summary ● Efficient Read ○ Read data from one Node due to BLS multi-signatures and state proofs ● Specific of the Protocol: ○ 3PC Batching ○ Data Consistency check as part of Consensus Protocol ○ Dynamic validation based on the current uncommitted state ○ Usage of Audit Ledger ○ Sequential applying of PrePrepares ○ BLS multi-signature as part of Consensus Protocol Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  40. 40. Links ● Plenum and Node: ○ https://github.com/hyperledger/indy-plenum/blob/master/README.md ○ https://github.com/hyperledger/indy-plenum/tree/master/docs/source ○ https://github.com/hyperledger/indy-node/blob/master/README.md ○ https://github.com/hyperledger/indy-node/tree/master/docs/source ● RBFT: ○ https://pakupaku.me/plaublin/rbft/5000a297.pdf ● Aardvark: ○ https://www.usenix.org/legacy/events/nsdi09/tech/full_papers/clement/clement.pdf ● PBFT: ○ https://www.microsoft.com/en-us/research/wp-content/uploads/2017/01/p398-castro-bft-tocs.p df Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  41. 41. https://www.manning.com/books/self-sovereign-identity (mlpreukschat - 50% off all formats until 31 December 2020) Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org
  42. 42. Hyperledger Indy Public Blockchain Presented by Alexander Shcherbakov Released under a Creative Commons license. (CC BY-SA 4.0). SSIMeetup.org

×