IBM Blockchain Platform - Architectural Good Practices v1.0

Architectural Good Practices
How to create blockchains that work well
V1.0, 4 October 2019
IBM Blockchain Platform Technical Series
Architectural Good Practices
Modeling Blockchain Applications
What’s New in Technology
Using IBM Blockchain Platform
Technical Introduction

Other Considerations
Optimizing Performance
GDPR ComplianceGDPR

What does performance
mean in the context of
blockchain?

A simple analogy to start with…
How would you maximize the throughput of a Ferris wheel?
https://www.maxpixel.net/Folk-Festival-Ferris-Wheel-Height-1024217
People = transactions Stalls = client apps
Gondolas/Pods = organizations

Recap on Hyperledger Fabric transaction process
Client
Application
SDK
Membership
Services
Peer
Endorser
Ledger
Committer
A
Chaincode B
!Events
Ordering-Service
O
O O
O
ü
Fabric-CA
ü
External-CA
Hyperledger Fabric Network
optionaloptional
Admin

So, what do we mean by performance?
Performance can mean anything… and in a de-centralized network the concept is even more vague.
Client App PeerOrg 1 PeerOrg 2 OrdererPeerOrg N
and what about the breaking point?
When transactions starts timing out…
Response time:
75 ms?
Endorsement: 10 ms
Committing: 5 ms
20 ms
10 ms
10 ms
5 ms
Ordering:
20 ms
Response time
or
system latency?
Endorsement
& validation
throughput ?
Endorsement
& validation
throughput ?
Endorsement
& validation
throughput ?
Block
throughput ?
Network latency?
Note: Response time figures are
not representative.

What are the Hyperledger Fabric performance levers?
A
B
0 1 2 3
Peer
Ledger
Blockchain WorldState
!Events
Chaincode
Channels
Local
MSP
1
4
3
2
1
3
4
• Indexing
• Size of payload
World State
Orderers / Channels (configtx.yaml)
• "absolute_max_bytes"
• "max_message_count"
• "timeout"
• World state queries
• Query aggregation (e.g. Sum)
• Business logic complexity
• Query to external sources (oracles)
Chaincode
2
• VSCC Transaction validation
• Peer’s resources
Committing peers
• ESCC Transaction signing
• Peer’s resources
Endorsing peers

What are the Client Application performance levers?
! Events
ChannelsClient
Application
SDK
(HFC)
Local
MSP
1
• GRPCs Connections reuse
• Endorsement load balancing
• Don’t always hit the same peer
• Complex Endorsement policies
• More endorsers = more invocations
• Channel Event Hubs impact
• How many events should you wait for?
• More peers an organization has on a
channel means more events to listen to
3
2
3
2
• Network latency
• To the orderer
• To the peers of every orgs
1 Fabric components
Fabric SDK / Client Application
Channels

Keeping up with the
community…
• 2018 has been an active year for
Blockchain and Hyperledger Fabric.
• Lots of work in the field of
performance has been delivered,
providing a constant stream of
optimization opportunity and new
ideas…
• As versions of the software evolve,
those reports are markers of a vibrant
community, however not every tuning
still applies…
https://arxiv.org/pdf/1808.08406.pdf

The context of blockchain performance
Performance must be seen in context of the wider system architecture
• What external applications submit transactions?
• Are there any SLAs?
• Can they process events successfully?

Getting started with
Hyperledger Fabric
performance testing

Performance testing requires a systemic
approach involving:
• Many test cycles
• High-volume of data to process
• Multiple configurations to support
Deployment
Tool
Data
Analysis
Load
Generator
Tool
System
Monitoring
Tool
Agility in
deployment
Efficient load
generation
From insight to
action
Capturing the
right data
Things to look for in a Blockchain performance tool
Pick the right tool!
Without the proper tools, progress will be slow...

13
Hyperledger
Performance Traffic
Engine (PTE)
https://github.com/hyperledger/fabric-
test/tree/master/tools/PTE
+ Multiple deployment
options (K8S, Cello, Cloud)
- Complex to manage and
configure
+ Aligned with Fabric
release
- Does not use your client
application
+ Basic metrics are
provided out of the box
- Missing advanced
metrics (ESCC, VSCC)
+ All the analysis you
can think of J
- Upfront investment
Hyperledger Caliper
https://hyperledger.github.io/caliper/
+ Pre-defined topologies
- Work required for
distributed network
topologies
+ Support distributed
clients
- Does not use your client
application
+ Basic metrics are
provided out of the box
- Limited support for
remote topologies
(Docker stats)
can think of J
Custom + Perfect for existing
project topologies
- Upfront time investment
to create topology
+ Complete flexibility in
generating the load
- Upfront time investment
to ensure your tool is
accurate
+ All the metrics you can
think of J
can think of J
Deployment Data analysis
Load
Generator
Monitoring
What are your options?

• Generic framework that supports many
blockchain technologies
• Benchmark layer defines the test cases
to invoke
• Blockchain (north bound interfaces) NBIs
define the integration point with the
underlying blockchain (e.g. chaincode
deploy)
• Resource Monitor currently supports:
• Local processes (e.g. client app)
• Remote/Local docker containers (e.g. peers)
Overview of Hyperledger Caliper
https://hyperledger.github.io/caliper/docs/2_Architecture.html

Structure of an Hyperledger Caliper project
The two things to define:
1. Network configuration
2. Benchmark tests
1
2

Output of a Hyperledger Caliper test run
https://hyperledger.github.io/caliper/assets/img/report.png

Use of custom performance test harness
• What if you want to test your Hyperledger Fabric client app?
• This is where a simple test harness can be useful!
Creates a series of promises
invoking your node.js client library
Capture execution time and
measure throughput
Using Node.js child process,
create has many clients are
required

• In Fabric v1.4 the peer and the orderer host an HTTP server that offers a RESTful
“operations” API that includes two ways to expose metrics:
• a pull model based on Prometheus (open-source monitoring solution)
• a push model based on StatsD (statistics aggregation daemon)
https://hyperledger-fabric.readthedocs.io/en/release-1.4/metrics_reference.html
Metric Name Description
endorser_propsal_duration The time to complete a proposal.
broadcast_validate_duration The time to validate a transaction in seconds
blockcutter_block_fill_duration
The time from first transaction enqueing to the block being
cut in seconds
ledger_blockstorage_commit_time Time for committing the block and private data to storage
ledger_statedb_commit_time Time for committing block changes to state db
couchdb_processing_time Time for the function to complete request to CouchDB
Leveraging the HL Fabric v1.4 metrics
It includes many
metrics that are
useful
in performance
testing.
Summary of some of the key metrics in performance testing:

What we learned so far
from real projects and
things to watch for…

1. Make sure you reuse your connections
2. Ensure your connection profile is
properly configured
3. Load balance endorsement requests
across multiple peers
4. Sample code is not a guarantee of
scalability
5. If starting a new project, leverage the
new HL Fabric v1.4 programming
model!
Optimizing the client application
Latency per tx when not
reusing client connections
1
Enabling the peers that need to endorse2

1. Understanding the effect of:
(configtx.yaml)
– Preferred Max Bytes
Represents the threshold at which a block will be
cut
– Block size
– Timeout
2. The peer and its dual personality
– Commit versus endorsement
– State validation is the most expensive operations
• Driven primarily by payload size and frequency
3. Increasing the number of endorsing
peers helps throughput
Transactions and the Orderer
Block commit State commit State validation

1. Test your queries against separate
CouchDB first
Use _explain and execution_stats to confirm
index is used
2. Avoid $or or $regexp in selectors
It ignores the index and forces CouchDB to
perform a full scan
3. Use pagination to improve performance
Pagination is supported since Hyperledger
Fabric v1.3
Considerations for queries

It’s all about the use-case
• Consider your non-functional requirements to drive the network topology
• There is no recipe about the “best” block size or the ideal number of peers
• Leverage the new fabric programming model to simplify your client application tuning

25
Integrating with Existing Systems – Possibilities
Transform
Existing
systems
1. System
events
2. Blockchain
events
4. Call out to existing systems
3. Call into blockchain network
from existing systems
Blockchain network Existing
systems
!
!

26
• Blockchain is a network system of record
• Two-way exchange
– Events from blockchain network create
actions in existing systems
– Cumulative actions in existing systems
result in Blockchain interaction
Integrating with Existing Systems – Using Middleware
• Transformation between blockchain and existing systems’ formats
– GBO, ASBO is most likely approach
– Standard approach will be for gateway products to bridge these formats
– Gateway connects to peer in blockchain network and existing systems
• Smart contracts can call out to existing systems
– Query is most likely interaction for smart decisions
• e.g. all payments made before asset transfer?
• Warning: Take care over predictability: transaction must provide same outputs each time it executes…

27
Non-determinism in blockchain
• Blockchain is a distributed processing system
– Smart contracts are run multiple times and in
multiple places
– As we will see, smart contracts need to run
deterministically in order for consensus to work
• Particularly when updating the world state
• It’s particularly difficult to achieve determinism with
off-chain processing
– Implement oracle services that are guaranteed to
be consistent for a given transaction, or
– Detect duplicates for a transaction in the
blockchain, middleware or external system
getDateTime()
getExchangeRate()
getTemperature()
random()
incrementValue
inExternalSystem(…)

28
Business Considerations
• As a B2B system, blockchain adds a number of aspects that are not typical in other projects:
– Who pays for the development and operation of the network?
– Where are the blockchain peers hosted?
– When and how do new participants join the network?
– What are the rules of confidentiality in the network?
– Who is liable for bugs in (for example) shared smart contracts?
– For private networks, what are the trusted forms of identity?
• Remember that each business network participant may have different requirements (e.g. trust)
– Evaluate the incentives of potential participants to work out a viable business model
• Mutual benefit → shared cost (e.g. sharing reference information)
• Asymmetric benefit → money as leveler (e.g. pay for access to KYC)

29
Trade-offs Between Non-Functional Requirements
Consider the trade-offs between
performance, security and resiliency!
Performance
o The amount of data being shared
o Number and location of peers
o Latency and throughput
o Batching characteristics
Security
o Type of data being shared, and with whom
o How is identity achieved
o Confidentiality of transaction queries
o Who verifies (endorses) transactions
Resiliency
o Resource failure
o Malicious activity
o Non-determinism

30
Non-Functional Requirements as a slider
Multi-tenant Cloud
Low Controls
Single Site Development
SaaS
Constrained
Monolithic
Higher re-use
Proven
Low
Low
Low
Dedicated
High Controls
High Availability & DR
On-Premise
Ready For Growth
Highly Modular
Custom Build
Leading Edge
High
High
High
Isolation
Security
Resilient Design
Dev Options
Sized For Growth
Componentisation
Re-Use
Production Readiness
Blockchain Network Complexity
Blockchain Network
Security/Privacy Complexity
Blockchain SC Complexity
Adjust the sliders with
the client early in the
project so all parties are
aligned on the
expectations of
robustness, isolation,
security controls etc. as
all these factors have
material impact on the
cost and complexity of
the solution.
$ $$$

31
Privacy & Confidentiality: Toolbox and Patterns
ü IBM Blockchain Platform delivers the
most comprehensive privacy
toolbox of any blockchain
technology provider.
ü IBM Research is the industry
leading expert behind these
technology advancements
ü Tools are often combined in patterns
to achieve the P&C requirements for
given industry data.
ü Many patterns are in use today in
production blockchain solutions:
TradeLens and IBM Food Trust, and
other IBM delivered solutions
ü All blockchain solutions have an
accompanied application stack. At
scale P&C has been a key focus
IBMs large program deliveries for
decades
.
ü Sharing needs of your industry
include public, dynamic, and secret
bilateral
Recommended Pattern for unobservable secret bilateral sharing
All parties ensure
verifiability without
knowing:
if activity is
real
Activity
Obfuscation
Noise
who is
acting
Identity
Obfuscation
IdMixer
transaction
details
Data
Isolation
Private Data
Collections

33
Summary
• Identify key operational considerations
• Consensus is the art of maintaining a consistent ledger
• It is possible to integrate with existing systems, but take care over determinism
• Security requirements solved through techniques such as encryption and signing
• Also consider business and non-functional requirements

35
GDPR Topics
• Summary of GDPR
• “Right to Erasure” and Blockchain
• Hyperledger Fabric Block
• Solution
• Non-Compliant Solutions
• Further Information

36
General Data Protection Regulation (GDPR) – Overview
• Employee or Customer
• New rights include:
– Information about processing
– Obtain access to PD
– Request corrections to PD
– Data be erased
– Object to being used for
marketing
– Restriction of processing
– Data portability
– Automated processing of PD
Data
Subject
Data Controller
Determines the
purpose and means
by which personal
data is processed
Contracts with
processor must be
GDPR compliant
Organisations can be either/both
Data Processor
Processes data on
behalf of data
controller
Legal Contract
specifies GDPR
duties to the controller
Any information
relating to an identified
or identifiable natural
person
Personal Data
• GDPR became law on 25th May 2018
• Applies to organisations operating within the EU, also applies to
organisations outside the EU who offer goods/services to EU individuals
• Fines of €20m or 4% of global revenues
The processor must provide sufficient guarantees
they will implement measures to meet GDPR.

37
General Data Protection Regulation (GDPR) – Links
• Rights for citizens
– https://ec.europa.eu/info/law/law-topic/data-protection/reform/rights-citizens/my-rights/what-are-my-rights_en
• What is Personal Data?
– https://ec.europa.eu/info/law/law-topic/data-protection/reform/what-personal-data_en
– Examples: Names, Address, Email, ID Card number, Location Data, IP Address, Cookie ID,
Advertising ID
– https://ico.org.uk/media/for-organisations/documents/1554/determining-what-is-personal-data.pdf
• What is a data controller or processor?
– https://ec.europa.eu/info/law/law-topic/data-protection/reform/rules-business-and-organisations/obligations/controller-processor/what-data-controller-or-data-processor_en
• Official Journal
– http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679&from=EN

38
Further Reading
• UK Information Commissioner’s Office (ICO) Data Protection Act (DPA) document that
helps determine if data is considered Personal Data. DPA is being superseded by GDPR:
– https://ico.org.uk/media/for-organisations/documents/1554/determining-what-is-personal-data.pdf
– DPA and GDPR
– https://www.itgovernance.co.uk/data-protection
– Anonymisation
• https://ico.org.uk/media/for-organisations/documents/1061/anonymisation-code.pdf
• NHS in the UK document on Anonymisation and Pseudonymisation Standard:
– https://www.nhsbsa.nhs.uk/sites/default/files/2018-
03/NHSBSA%20Anonymisation%20and%20Pseudonymisation%20Standard.pdf
• WP216 opinion 05/2014 on Anonymisation Techniques (often cited in GDPR information):
– https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp216_en.pdf

39
GDPR Topics
• Summary of GDPR
• Solution

40
The “Right to Erasure” and blockchain
Enables an individual to request the deletion or removal of their personal data
Config
Block
0
Config
Block
1
Transaction
Block
2
Transaction
Block
3
Genesis
Worldstate
Blockchain
Transactions in
the blockchain
are immutable
Data in the
world state is
mutable

41
Further Reading
• UK Information Commissioner’s Office (ICO) “Right to Erasure” guidance:
– https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/right-to-erasure

42
GDPR Topics
• Summary of GDPR
• Solution

43
Summary of a Hyperledger Fabric Block
Header Number Previous hash (merkel root) Data hash (merkel root)
BlockData
(repeatedpertransaction)
Signature Client signature of transaction payload
Payload Header Channel Header (timestamp, tx_id, type, …) Signature Header (client certificate)
Data Client certificate MspId
Proposal payload Proposal hash
Array of keys read (key and version of value)
Array of keys written (key/value and delete flag)
Array of range_queries (start/end key and list)
Events (chaincode_id, tx_id, name, payload)
Chaincode response (status, message, payload)
Chaincode ID (path, name, version)
Array of endorsers (mspid, certificate, signature)
Block Metadata
Array of orderers (mspid, certificate, signature)
Array of transaction valid flags
• Fields highlighted in red could potentially contain Personal Data recorded on the blockchain.
DataIndex
Block

44
Block Fields and Personal Data
This slide describes the potential areas that Personal Data could be included in the block:
• Proposal payload : These are the transaction input arguments.
• Client certificate: If the client certificate is individual to a Data Subject then it could be
considered PD.
• Key: The key written to or read from the world state. If PD is used in the key name then it will be
included here.
• Value: The value of any keys written to the world state. If PD is used in the key value then it will
be included here.
• Events: Events emitted from chaincode could include PD.
• Chaincode response: Any response from invoking the chaincode could include PD.
It is assumed that endorser and orderer certificates are issued to an organization and do not
include Personal Data.
Remember that Fabric configuration blocks also contain certificates!

45
GDPR Topics
• Summary of GDPR
• Solution

46
Solution – Store data off-chain
Config
Block
0
Config
Block
1
Transaction
Block
2
Transaction
Block
3
Genesis
Worldstate
Blockchain
Data Store
Salted hash
stored on-chain
matches hash in
data store
• What
– Store Personal Data (PD) in an off-chain mutable data storage
• How
– Store only proofs as a salted hash of the PD on-chain
– The salt must be stored securely, and should be unique for each PD
– The salted hash can be stored in an off-chain data store and linked to the PD
– Salt and salted hash can be in different data stores to the PD
– Warning: Hashes and salted hashes are considered pseudonymous data and
still fall under the scope of GDPR.
• Why
– PD, salted hash and salt can be deleted at any point from the data store(s)
causing the hash on-chain to be anonymized
Personal Data
Salt
salted hash
salted hash
salted hash

47
Notes – Store data off-chain
• Alternative solutions
⎼ There are alternative solutions that balance the right-to-erasure with
checking the integrity of the PD. For example a crypto random number
(CRN) is linked with the PD both of which are stored in an off-chain data
store. The CRN is then stored on-chain. The CRN is not computed from
the value of the PD (as in the case of a hash) and is perhaps easier to
anonymise, but doesn’t provide proof of the actual value of PD.
⎼ It is best practice to associate a CRN with a single PD record
• Reidentification of data
– Check the risk of reidentification of the data stored on-chain. For example
transaction correlation or improper controls that allows someone to
associate the data stored on-chain with the off-chain PD.
• It is important to consult with IBM Security counsel on any solution.

48
GDPR Topics
• Summary of GDPR
• Solution

49
Store Personal Data in Private Data Collections
Config
Block
0
Config
Block
1
Transaction
Block
2
Transaction
Block
3
Genesis
Worldstate
Blockchain
Hashes of PD stored
on blockchain
PD passed in the
transient data field and
stored in the private
world state and private
writeset storage
• What
– Send Personal Data in transient part of blockchain transaction, save
PD in private state and hashes on the blockchain
• How
– PD is sent in the transient data field to the Smart Contract (FAB
2450), data stored in private state, hashes are stored on the
channel and writeset is stored in the private writeset storage (FAB
1151). Available in Fabric v1.2.
• Why not compliant
– FAB 1151 stores the private data writeset in storage similar to a
blockchain. This private writeset storage cannot be deleted (other
than a blockToLive policy on the collection) and therefore a request
to erase PD cannot be processed.
Private
State

50
Private Data Collections : Further Information
• Further enhancements to Private Data Collections in Fabric will enable data in the private
writeset and world state to be deleted on demand. This may make it suitable for storing
personal data within the bounds of GDPR regulation. Currently there is no timeframe for
this additional feature. Further information can be found here:
– https://jira.hyperledger.org/browse/FAB-5097

51
Encrypt data
Config
Block
0
Config
Block
1
Transaction
Block
2
Transaction
Block
3
Genesis
Worldstate
Blockchain
All PD is encrypted
before being added
to blockchain
Private Key
Public Key
Data Store• What
– Encrypt Personal Data before being stored on blockchain
• How
– Public/Private Keys must be secured in a mutable data store
• Why not compliant
– Unproven for compliance with GDPR. Encrypted data is
considered pseudonymous data, and introduces complexity of
key management.

52
GDPR Topics
• Summary of GDPR
• Solution

53
Further Information
• GDPR consideration for blockchain solution architects
– https://www-01.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=84018584GBEN&
• Blockchain and GDPR
– https://www-01.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=61014461USEN

Thank you
www.ibm.com/blockchain
developer.ibm.com/blockchain
www.hyperledger.org
© Copyright IBM Corporation 2019. All rights reserved. The information contained in these
materials is provided for informational purposes only, and is provided AS IS without warranty
of any kind, express or implied. Any statement of direction represents IBM's current intent, is
subject to change or withdrawal, and represents only goals and objectives. IBM, the IBM
logo, and other IBM products and services are trademarks of the International Business
Machines Corporation, in the United States, other countries or both. Other company, product,
or service names may be trademarks or service marks of others.

IBM Blockchain Platform - Architectural Good Practices v1.0

IBM Blockchain Platform - Architectural Good Practices v1.0

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to IBM Blockchain Platform - Architectural Good Practices v1.0

Similar to IBM Blockchain Platform - Architectural Good Practices v1.0 (20)

More from Matt Lucas

More from Matt Lucas (9)

Recently uploaded

Recently uploaded (20)

IBM Blockchain Platform - Architectural Good Practices v1.0