This presentation cuts through the confusion and hype surrounding blockchain, explains the key technical aspects of blockchain systems, and summarizes the point of view of different blockchain luminaries and open sourced systems. Originally presented as a guest lecture at Columbia University in April 2019.

1. Blockchain – Beyond the Hype
Salman Baset
April 2nd, 2019

2. My goals for this talk…
• Blockchain is a new and emerging technology. As it is the case with any emerging technology,
there are claims and counter-claims on competing technologies, and their applications. Generally
speaking, there is a lot of confusion and lots of buzz words get thrown around.
• Eventually clarity will emerge, given time and effort by folks (remember OSI vs TCP/IP?)
• My goal is
• (1) to not make you more confused than you already were about blockchain
• (2) to help cut through confusion through examples, definitions, concepts, and solution examples.
2

3. Outline
• What is trust?
• What is blockchain?
• Design options for blockchain
• CAP theorem, perspectives of key folks in blockchain space
• Consensus algorithms
• Hyperledger
• Blockchain applications
• Hyperledger internships
3

4. “We have proposed a system for electronic
transactions without relying on trust.”
4

5. “We have proposed a system for electronic
transactions without relying on trust.”*
* https://bitcoin.org/bitcoin.pdf
Not true – says Bruce Schneier – Hyperledger Forum, December 2018
5

6. What is trust? (1/2)
Reference: Bruce Schneier’s example from Hyperledger Forum 2018
My building
lobby
(1)
(2) (3)
(4)
6

7. What is trust? (2/2)
• Can trust ever be eliminated?
• Trust is not eliminated. It is shifted.
• In the case of Bitcoin, the trust shifts from Governments and Banks to
a piece of software.
7

8. Bitcoin “paper” – Conclusion
We have proposed a system for electronic transactions without relying on trust.
We started with the usual framework of coins made from digital signatures, which
provides strong control of ownership, but is incomplete without a way to prevent
double-spending. To solve this, we proposed a peer-to-peer network using proof-
of-work to record a public history of transactions that quickly becomes
computationally impractical for an attacker to change if honest nodes control a
majority of CPU power. The network is robust in its unstructured simplicity. Nodes
work all at once with little coordination. They do not need to be identified, since
messages are not routed to any particular place and only need to be delivered on a
best effort basis. Nodes can leave and rejoin the network at will, accepting the
proof-of-work chain as proof of what happened while they were gone. They vote
with their CPU power, expressing their acceptance of valid blocks by working on
extending them and rejecting invalid blocks by refusing to work on them. Any
needed rules and incentives can be enforced with this consensus mechanism.
https://bitcoin.org/bitcoin.pdf 8

9. Bitcoin “paper” – Conclusion
We have proposed a system for electronic transactions without relying on trust.
We started with the usual framework of coins made from digital signatures, which
provides strong control of ownership, but is incomplete without a way to prevent
double-spending. To solve this, we proposed a peer-to-peer network using proof-
of-work to record a public history of transactions that quickly becomes
computationally impractical for an attacker to change if honest nodes control a
majority of CPU power. The network is robust in its unstructured simplicity. Nodes
work all at once with little coordination. They do not need to be identified, since
messages are not routed to any particular place and only need to be delivered on a
best effort basis. Nodes can leave and rejoin the network at will, accepting the
proof-of-work chain as proof of what happened while they were gone. They vote
with their CPU power, expressing their acceptance of valid blocks by working on
extending them and rejecting invalid blocks by refusing to work on them. Any
needed rules and incentives can be enforced with this consensus mechanism.
https://bitcoin.org/bitcoin.pdf
So, no non-electronic (aka paper) transactions? enuf said.
do not pay more than once really? what is work?
for ever?
distributed? anonymity is a goal ?
churn consensus algorithm
is fixed.
interesting the core …
9

10. The origins of the word blockchain…
Ehrsam, Meyer, Smith and Tuchman - Cipher Block Chaining (CBC) mode of operation - 1976
https://patents.google.com/patent/US4074066A/en
https://satoshi.nakamotoinstitute.org/emails/cryptography/6/
Hal Finney – “block chain” – two letter word. 2008-11-09 01:58:48 UTC
Earliest…
As recently
understood…
(approx.)
10

11. The emergence of “standard” definitions to buzz word terms…
• What is a standard definition?
• A definition created by a standards body such as National Institute of Standards and Technology (NIST) – US, or ISO.
The definition is typically first published as a draft version for feedback from public at large or a select group.
• Cloud Computing
• Utility Computing (McCarthy – 1961), Cloud computing (1996, Compaq Business plan per MIT TR), AWS 2006
• NIST.SP.800-145 - November 2009 (first draft), October 2011 (published).
• https://www.nist.gov/news-events/news/2011/10/final-version-nist-cloud-computing-definition-published
• ISO – ISO/IEC 17788:2014
• https://www.iso.org/standard/60544.html
• Blockchain (~2008)
• NIST.IR.8202 - Blockchain Technology Overview – January 2018 (first draft), October 2018 (published)
• https://csrc.nist.gov/publications/detail/nistir/8202/final
• ISO definition still in progress
• https://www.iso.org/standard/73771.html?browse=tc
11

12. What is blockchain?
• NIST.IR.8202
• (first two lines of intro) Blockchains are tamper evident and tamper resistant
digital ledgers implemented in a distributed fashion (i.e., without a central
repository) and usually without a central authority (i.e., a bank, company, or
government). At their basic level, they enable a community of users to record
transactions in a shared ledger within that community, such that under
normal operation of the blockchain network no transaction can be changed
once published.
What is a block? – grouping of transactions
What is a transaction? – a state changing mechanism
Does the definition clearly state append-only aspect of ledger?
12

13. What is a “ledger”?
• https://www.merriam-webster.com/dictionary/ledger
• a book containing accounts to which debits and credits are posted from books
of original entry
• a horizontal board used for vertical support (as in scaffolding)
• The ledger familiar to “most” of us…?
• Personal journal
13

14. Ledger vs. personal journal
Ledger Personal journal
Written on Paper with typically pen Paper, with pen or pencil
Can also be written with IT system (e.g., computer, SaaS) IT system (e.g., computer, SaaS)
Record of who made changes Important Not so much
Common primary application Recording monetary
transactions
Thoughts
Information lay out Structure (tabular), credit/debit,
with dates
Usually with dates
Information is appended? Typically, yes Typically, yes
Shared with others Employees (probably). Other
entities, no, unless IRS J
Depends J
14

15. What is a digital ledger?
• A ledger stored in a digital form
• On a (personal) computer or a set of computers
• Can contain data ranging from few bytes to peta bytes, and beyond
• What is distributed paper ledger?
• Create copies of paper and distribute it to relevant folks whenever there is a change?
• What is a distributed digital ledger? (or simply distributed ledger)
• Ledgers stored in digital form on a set of computers (e.g., cloud), where data
repository is not confined to a single computer (NIST: without central repository).
The structure of the information stored within the ledger depends on the
application.
15

16. Tamper evident and tamper resistant
• Immutable: Merriam-Webster
• https://www.merriam-webster.com/dictionary/immutable
• not capable of or susceptible to change
• Why do we write personal checks with a pen and not with a pencil?
• May be, no one writes personal checks these days J
• tamper evident and tamper resistant – to an extent
• Can a (distributed) digital ledger be changed?
• Of course!
• How to detect changes to a (distributed) digital ledger and prevent changes?
• Detect changes: audit logs
• Prevent changes: authz, authn (requires identity)
16

17. [Lack of] Central authority or central repository - Examples
• I run a database on my single machine.
• Central repository?
• Central authority?
• A big search engine has a massive farm of distributed machines connected over network, that work
together to respond to search queries.
• Central repository?
• Central authority?
• A music file-sharing system (Napster) has a central list of which users have files, but files are
downloaded peer-to-peer.
• Central authority?
• Central repository?
• A file-sharing network has a distributed index of files and file chunks.
• Central repository?
• Central authority?
17

18. Lets look at Bitcoin again
1. Decentralized – without central
authority?
• Hash power concentration
• https://www.blockchain.com/en/pools
• Crypto exchange concentration
• https://www.bbc.com/news/world-us-
canada-47203706
2. Distributed?
• Yeah
3. Anonymity?
• Public key per transaction
4. Energy efficiency?
• No
4. Non-crypto currency applications
• Not easy
5. Storage capacity per node (~30 GB)
• Ever increasing
6. Mining reward
• Decreasing with time. 12.5 bitcoins today
7. Transaction fees
• How a transaction ends up in a block
• Increasing with time
• Resistance to Sybil attacks
• Sybil attack: Create ‘fake identities’ to take
over portion of id space in a system
• Resistance: Yes, proof of work
18

19. Bitcoin: transaction types
Input 0 from
Alice, signed
by Alice
Output 0
“To Bob”
Output 1
“To Alice”
(change)
Common transaction
Alice pays Bob
1
2 Input 0
Output 0
Transaction aggregating funds
Input 1
Input N
3
Transaction distributing funds
Input 0
Output 0
Output 1
Output N
Payment is to a “public key”
19
Output => UTXO: unspent transaction output

20. Bitcoin: transactions are grouped into blocks
Block N
T1
T2: A -> B
T3
T4
Block N+1
B->C
Block N+2
C->D
H(N) H(N+1)
T1 T2 T3 T4
H() H() H() H()
H() H()
H(N) Mining: find a nonce such hash of merkle tree + nonce
is less than target and takes on avg 10 minutes to compute
Which transaction to include in block? Transaction fee higher
than some minimum. [decentralized or distributed?]
20

21. CAP theorem…
https://cryptographics.info/cryptographics/blockchain/cap-theorem/
21

22. The design options in Blockchain
• Scalability – as many nodes as possible
• Performance – write to chain as quickly as possible
• Anonymity (or conversely, strong identity) – not “easily traceable” back to entity
• Privacy – keep transactions data and transaction occurrence confidential among
involved parties
• Often informed by:
• Application-specific (crypto-currency) vs. agnostic
• One network to rule them all vs. many networks
• Environmental friendly (Bitcoin proof-of-work is energy inefficient)
22

23. The design options in Blockchain –
Scalability Triangle – Vitalik Buterin
Assume the total computational/bandwidth capacity of a
regular computer is O(c), and the total load of a blockchain is
O(n)
Decentralization: the system can run in an environment where
all nodes have O(c) resources
• Possible weakening: can have supernodes, but require only 1 of N supernodes to be
honest
Security: the system can survive attacks up to some specific
percentage of all
miners/validators (eg. 33%)
Scalability: the system can handle a load of O(n) > O(c)
• Computation
• State storage
• Bandwidth
https://vitalik.ca/files/Ithaca201807_Sharding.pdf
Scalability
Security
Decentralization
Claim: we can reach the middle of the triangle, though we do need to use some more complex tools to get
there (slide 6 of link below)
23

24. The design options in Blockchain –
Matt Corello – aka Bluematt - Bitcoin
• Trustlessness
• Scalability
• Identity (or lack of it)
http://diyhpl.us/wiki/transcripts/baltic-honeybadger/2018/trustlessness-scalability-and-directions-in-security-models/
24

25. The design options in Blockchain –
Hyperledger architecture group white paper
• Linux foundation project for creating open source permissioned blockchain networks
https://www.hyperledger.org/wp-content/uploads/2017/08/Hyperledger_Arch_WG_Paper_1_Consensus.pdf
Comparison of permissioned consensus approaches and standard proof of work
25

26. What is Finality?
• “Finality means that once a transaction is committed, it cannot be
reversed, i.e. the data cannot be rolled back to the previous state.
Different blockchain systems may provide different types of finality.
Typically, this is defined in the consensus protocol. Different types of
consensus exist, such as voting-based consensus with immediate
finality and lottery-based consensus with probabilistic finality.”
https://www.hyperledger.org/wp-content/uploads/2018/10/HL_Whitepaper_Metrics_PDF_V1.01.pdf
26

27. The design options in Blockchain –
CAP theorem equivalent for blockchain
• Decentralization, consistency, scalability
• http://msrg.org/publications/pdf_files/2018/bcbi-icdcs18-
Towards_Dependable,_Scalable,_an.pdf
27

28. What is CAP-theorem-equivalent for blockchain?
• Which blockchain?
• Which consensus algorithm?
• Which model?
• …
Will be a nice contribution….
28

29. Consensus algorithms
• Background
• Crash-tolerant consensus
• Byzantine consensus
• Nakamoto consensus (aka proof of work)
• Proof of stake
• Proof of elapsed time
https://hackernoon.com/a-hitchhikers-guide-to-consensus-algorithms-d81aae3eb0e3
Cachin, Vucolic - https://arxiv.org/pdf/1707.01873.pdf
29

30. Consensus algorithms - Background
• Lamport Byzantine generals problem [1982]
• Schneider – task of reaching and maintaining consensus among nodes
involves [1990]:
• A (deterministic) state machine that implements the logic of the service to be
replicated
• A consensus protocol to disseminate requests among the nodes, such that
each node executes the same sequence of requests on its instance of the
service.
• Traditionally in literature, consensus means reaching agreement on
single request (first) – atomic broadcasts provide agreement on
sequence of requests.
Cachin, Vucolic - https://arxiv.org/pdf/1707.01873.pdf
30

31. Crash-tolerant consensus
• Survive node crashes
• Progression in epochs or views with a unique leader for each view
• Replace leader
• N/2-1 failures
• Paxos (Zoo-keeper), Raft (etcd)
31

32. Byzantine consensus
• Practical Byzantine Fault Tolerance (PBFT)
• < n/3 (note the inequality) nodes can be malicious (or system must
have 3k+1 nodes for k nodes to be malicious)
• Commercial implementations hard to find
32

33. Nakamoto consensus aka proof of work
• (Nodes) vote with their CPU power, expressing their acceptance of
valid blocks by working on extending them and rejecting invalid
blocks by refusing to work on them. Any needed rules and incentives
can be enforced with this consensus mechanism.
33

34. Proof of stake
• Miners stake their tokens on which blocks are valid
• Miners spend their tokens on which fork to support
• If most people vote on the correct fork, miners who work on the
wrong fork can loose their stake
34

35. Distributed Ledger Technologies aka Blockchain categorization
Drive value of cryptocurrency Cryptocurrency for a business use-case
Blockchain for business
Anonymous Permissioned
CryptocurrencyNon-Cryptocurrency
Standards bodies and consortiums
35

36. Blockchain types
• Public blockchain
• Ledger is public, anyone can join?
• Private blockchain
• Ledger is private, anyone can join?
• Permissioned blockchain
• Ledger can be public or private, join requires consent of existing parties
36

37. What is Hyperledger?
• Hyperledger is an open source collaborative effort created to advance cross-industry blockchain
technologies. It is a global collaboration, hosted by The Linux Foundation, including leaders in finance,
banking, Internet of Things, supply chains, manufacturing and Technology.
• Launched in February 2016
https://www.hyperledger.org/about
Frameworks
Tools
Hyperledger
Indy
Hyperledger
Fabric
Hyperledger
Iroha
Hyperledger
Sawtooth
Hyperledger
Burrow
Hyperledger
Composer
Hyperledger
Explorer
Hyperledger
Cello
37

38. Hyperledger Projects Design Philosophy
• Consensus Layer
• Responsible for generating an agreement on the order and confirming the correctness of the set of transactions that constitute a block
• Smart Contract Layer
• Responsible for processing transaction requests and determining if transactions are valid by executing business logic.
• Communication Layer
• Responsible for peer-to-peer message transport between the nodes that participate in a shared ledger instance
• Data Store Abstraction (Ledger)
• Allows different data-stores to be used by other modules.
• Crypto Abstraction
• Allows different crypto algorithms or modules to be swapped out without affecting other modules.
• Identity Services
• Enables the establishment of a root of trust during setup of a blockchain instance, the enrollment and registration of identities or system entities during network
operation, and the management of changes like drops, adds, and revocations. Also, provides authentication and authorization.
• Policy Services
• Responsible for policy management of various policies specified in the system, such as the endorsement policy, consensus policy, or group management policy. It
interfaces and depends on other modules to enforce the various policies.
• APIs
• Enables clients and applications to interface to blockchains.
• Interoperation
• Supports the interoperation between different blockchain instances
38

39. Hyperledger Fabric releases
• https://github.com/hyperledger/fabric#releases
39

40. Overview of Hyperledger Fabric – Key Design Goals
• The four P’s
• Permissioned
• Privacy
• Pluggability
• Performance
40

41. Permissioned: Existing members determine who can join the network,
and update configuration
Public blockchains
• Download software and connect to network
• Configuration updated through developer
or community consensus
Hyperledger Fabric
• Policy-based mechanism to admit new
members and to update configuration
6/8 votes
(admit A: majority vote)
A
A
B
I want to invite A to network
A
B
A
B
A
B A
B
A
B
A
B
A
B
B
I want to invite B to network
A
B
3/8 votes
(reject B: majority vote)
Permissioned != Private 41

42. Privacy: Smart contract execution, and transaction data storage limited to a
set of nodes in the network based on policy
Public blockchains
• Every node runs smart contract
• Every full node can potentially have a full copy
of the ledger
Hyperledger Fabric
• A subset of nodes will run smart contracts
• The ledger updates are limited to set of
nodes (channel).
• Nodes in a channel can directly share private
data with subset of nodes (collections,
v1.1 feature)
SC
SC
SC
SC
SC
SC
SC
SC
SCSC
SC
Channel /
Private data collection
42

43. Pluggability: consensus, identity provider, crypto, data format, smart
contract language
Public blockchains
• Fixed or hard to change consensus algorithm (proof of work)
• Fixed encryption (e.g., secp256k1)
• Identity = public key - self
• Domain specific language (DSL) for writing smart contracts
Hyperledger Fabric
• Pluggable consensus algorithm (PBFT, Kafka)
• Pluggable crypto service provider
• Pluggable identity provider, zero knowledge
proofs
• General data format, key / value pair
• General purpose languages for writing smart contracts
(Javascript, Go)
43

44. Performance: transactions should commit quickly
Public blockchains
• Use proof of work / proof-of-stake as consensus
algorithm due to which it may be a while before
transactions are final (e.g., 60 minutes)
• Transaction commitments are often
probabilistic
Hyperledger Fabric
• Uses a voting based consensus mechanism
• Transaction commitments are immediately final
44

45. Ok, so where is mining in Hyperledger Fabric?
• There is no mining – however, the consensus algorithm is pluggable
• Consensus properties: safety, liveness
• The Kafka service used in Hyperledger Fabric is crash tolerant, but not Byzantine fault tolerant
source: https://www.hyperledger.org/wp-content/uploads/2017/08/Hyperledger_Arch_WG_Paper_1_Consensus.pdf
Hyperledger Fabric
Hyperledger Sawtooth
45

46. Hyperledger Fabric and Ethereum* – A summary
Hyperledger Fabric Ethereum
Open source Yes Yes
Live network No – create your own network Yes
Who can join the network? Depends on the rules created by network
creators – default majority vote (Permissioned)
Anyone
Smart contract language Javascript, Go
Smart contracts called chaincode in Fabric
Solidity, Vyper, LLL
Rogue smart contract prevention Explicit review and installation of smart code
before execution (no notion of gas)
Gas
Who executes smart contracts? Subset of nodes (or all) All nodes by default / channels
Who stores the blocks? (state visibility) Subset of nodes (or all) All nodes by default / channels
Consensus algorithm Execute-order-validate (voting) – no mining PoW, PoS
Identity X.509 certificate issued by membership service
provider of a peer. Contains public key
Public key
Incentive for running a node Business network Ether
*comparing apples to oranges? (open source software vs live public network)
46

47. Blockchain Applications
• Cryptocurrency (enuf said)
• Food Safety (provenance of goods from source to destination)
• The physical to digital correlation problem…
• Vitalik’s view on Blockchain applications (Dec 2018) tweet
• 50 examples of blockchain real world use cases
• Self-“Sovereign” Identity
47

48. Food Safety
Seller Crop / Quantity Buyer
Joe (farmer) sold crops (SKU#j1) Mangoes / 100 lbs Alice
Alice (distributor) sold crops (SKU#j1) Mangoes / 10 lbs Walmart
Alice (distributor) sold crops (SKU#j1) Mangoes / 90 lbs Kroger
Tina (farmer) sold crops (SKU#t1) Mangoes / 200 lbs Walmart
‘Joe’ the
farmer
‘Alice’ the distributor
“Tina” the
farmer
Seller Crop/Quantity Buye
r
Joe(farmer)soldcrops(SKU#j1) Mangoes/100lbs Alice
Alice(distributor)soldcrops(SKU#j1) Mangoes/10lbs Wal
mar
t
Alice(distributor)soldcrops(SKU#j1) Mangoes/90lbs Krog
er
Tina(farmer)soldcrops(SKU#t1) Mangoes/200lbs Wal
mar
t
Seller Crop/Quantity Buye
r
Joe(farmer)soldcrops(SKU#j1) Mangoes/100lbs Alice
Alice(distributor)soldcrops(SKU#j1) Mangoes/10lbs Wal
mar
t
Alice(distributor)soldcrops(SKU#j1) Mangoes/90lbs Krog
er
Tina(farmer)soldcrops(SKU#t1) Mangoes/200lbs Wal
mar
t
Seller Crop/Quantity Buye
r
Joe(farmer)soldcrops(SKU#j1) Mangoes/100lbs Alice
Alice(distributor)soldcrops(SKU#j1) Mangoes/10lbs Wal
mar
t
Alice(distributor)soldcrops(SKU#j1) Mangoes/90lbs Krog
er
Tina(farmer)soldcrops(SKU#t1) Mangoes/200lbs Wal
mar
t
Seller Crop/Quantity Buye
r
Joe(farmer)soldcrops(SKU#j1) Mangoes/100lbs Alice
Alice(distributor)soldcrops(SKU#j1) Mangoes/10lbs Wal
mar
t
Alice(distributor)soldcrops(SKU#j1) Mangoes/90lbs Krog
er
Tina(farmer)soldcrops(SKU#t1) Mangoes/200lbs Wal
mar
t
Seller Crop/Quantity Buye
r
Joe(farmer)soldcrops(SKU#j1) Mangoes/100lbs Alice
Alice(distributor)soldcrops(SKU#j1) Mangoes/10lbs Wal
mar
t
Alice(distributor)soldcrops(SKU#j1) Mangoes/90lbs Krog
er
Tina(farmer)soldcrops(SKU#t1) Mangoes/200lbs Wal
mar
t
Block N
Block N+1
Block N+2
Block N+3
48

49. Vitalik’s view on blockchain applications
• https://twitter.com/VitalikButerin/status/1072162014498148355
49

50. 50+ examples of blockchains in real world
https://medium.com/@matteozago/50-examples-of-how-blockchains-are-taking-over-the-world-4276bf488a4b
50

51. Self-sovereign identity (SSI)
• What is SSI? (def. by Christopher Allen)
• http://www.lifewithalacrity.com/2016/04/the-path-to-self-soverereign-identity.html
• The principles
• Existence. Users must have an independent existence.
• Control. Users must control their identities.
• Access. Users must have access to their own data.
• Transparency. Systems and algorithms must be transparent.
• Persistence. Identities must be long-lived.
• Portability. Information and services about identity must be transportable.
• Interoperability. Identities should be as widely usable as possible.
• Consent. Users must agree to the use of their identity.
• Minimalization. Disclosure of claims must be minimized.
• Protection. The rights of users must be protected.
• Decentralized identity foundation (https://identity.foundation/)
• Sovrin foundation (https://sovrin.org/) – Columbia can be a steward node
(https://sovrin.org/stewards/)
51