Blockchain Technology Basics Course

MODULE 1: BLOCKCHAIN
TECHNOLOGY BASICS
BLOCKCHAIN TECHNOLOGY
COURSE CODE: 21MCS25B2
1

A BRIEF HISTORY OF BLOCKCHAIN
• On October 31, 2008, Satoshi Nakamoto released the Bitcoin White
Paper outlining a purely peer to peer electronic cash/digital asset
transfer system. This is the first popular implementation of Blockchain
and is attributed as birthing today’s Blockchain industry. Since then,
additional Blockchains have been popularized, Ethereum, various
Hyperledger project solutions, as well as numerous others including
“Blockchain like” solutions such as GuardTime’s KSI products
2

“BLOCKCHAIN” HAS MANY MEANINGS
3
“T
o understand the power of blockchain systems,and the things they can do,it is
important to distinguish between three things that are commonly muddled up,namely
the bitcoin currency,the specific blockchain that underpins it and the idea of
blockchains in general.
”
TheT
rust Machine,THE ECONOMIST, Oct. 31,2015

WHAT IS BLOCKCHAIN?
• Blockchain is a system comprised of..
• Transactions
• Immutable ledgers
• Decentralized peers
• Encryption processes
• Consensus mechanisms
• Optional Smart Contracts
• Let’s explore these concepts
4

WHAT IS BLOCKCHAIN?
5
A technology that:
permits transactions to be
gathered into blocks and recorded;
allows the resulting ledger to be
accessed by different servers.
cryptographically chains blocks
in chronological order; and

WHAT IS A DISTRIBUTED LEDGER?
6
Centralized Ledger Distributed Ledger
Bank
ClientA
Client
C
Client D
Client
B
NodeA
Node B
Node C
Node D
Node E
• There are multiple ledgers,but Bank holds the“golden record”
• Client B must reconcile its own ledger against that of Bank,and
must convince Bank ofthe“true state” ofthe Bank ledger if
discrepancies arise
• There is one ledger. All Nodes have some level of access to that
ledger.
• All Nodes agree to a protocol that determines the “true state” of
the ledger at any point in time. The application of this protocol is
sometimes called“achieving consensus.
”

WHAT ISA DISTRIBUTED LEDGER?
Single Entity Multiple Entities
7

TRANSACTIONS
• As with enterprise transactions today, Blockchain is a historical archive of
decisions and actions taken
• Proof of history, provides provenance
• Demo - https://anders.com/blockchain/blockchain.html
8
Notable transaction use cases
Land registration – Replacing requirements for research of Deeds (Sweden Land Registration)
Personal Identification – Replacement of Birth/Death certificates, Driver’s Licenses, Social Security Cards (Estonia)
Transportation – Bills of Lading, tracking, Certificates of Origin, International Forms (Maersk/IBM)
Banking – Document storage, increased back office efficiencies (UBS, Russia’s Sberbank)
Manufacturing – Cradle to grave documentation for any assembly or sub assembly
Food distribution – Providing location, lot, harvest date Supermarkets can pin point problematic food (Walmart)
Audits – Due to the decentralized and immutable nature of Blockchain, audits will fundamentally change.

IMMUTABLE
• As with existing databases, Blockchain retains data via transactions
• The difference is that once written to the chain, the blocks can be
changed, but it is extremely difficult to do so. Requiring rework on all
subsequent blocks and consensus of each.
• The transaction is, immutable, or indelible
• In DBA terms, Blockchains are Write and Read only
• Like a ledger written in ink, an error would be be resolved with
another entry
9

DECENTRALIZED PEERS
• Rather than the centralized “Hub and Spoke” type of network, Blockchain
is a decentralized peer to peer network. Where each NODE has a copy of
the ledger.
Legacy Network Blockchain Network
Centralized DB Distributed Ledgers
10

ENCRYPTION
• Standard encryption practices
• Some Blockchains allow for “BYOE” (Bring Your Own Encryption)
• Only as good as the next hardware innovation
• All blocks are encrypted
• Some Blockchains are public, some are private
• Public Blockchains are still encrypted, but are viewable to the public, e.g.
https://www.blocktrail.com/BTC
• Private Blockchains employ user rights for visibility, e.g.
• Customer – Writes and views all data
• Auditors – View all transactions
• Supplier A – Writes and views Partner A data
• Supplier B – Writes and views Partner B data
11

CONSENSUS
• Ensures that the next block in a blockchain is the one and only version of
the truth
• Keeps powerful adversaries from derailing the system and successfully
forking the chain
• Many Consensus mechanisms, each with pros and cons
12
Consensus Mechanism
Proof of Work
Proof of State
Proof of Elapsed Time
Proof of Activity
Proof of Burn
Proof of Capacity
Proof of Importance
And others….

SMART CONTRACTS
• Computer code
• Provides business logic layer prior to block submission
13
Blockchain Smart Contracts? Language
Bitcoin No
Ethereum Yes Solidity
Hyperledger Yes Various GoLang, C++,
etc, depends
Others Depends Depends

BLOCKCHAIN CAPABILITIES
14
A shared ledger technology allowing any
participant in the business network to see
the system of record (ledger)
Ensuring appropriate visibility; transactions
are secure, authenticated & verifiable
Business terms embedded in
transaction database & executed with
transactions
All parties agree to network verified
transaction
Blockchain Essentials
1. A business problem to be solved
• That cannot be solved with more mature
technologies
2. An identifiable business network
• With Participants, Assets and
Transactions
3. A need for trust
• Consensus, Immutability, Finality or
Provenance
Negative Indicators, Anti-Patterns
1. Need high performance (millisecond) transactions
2. Small organization (no business network)
3. Looking for a database replacement
4. Looking for a messaging replacement
5. Looking for transaction processing replacement
6. Process and metrics are not clear within the
ecosystem
7. Value, velocity and/or variability are not present

HOW MIGHT A DISTRIBUTED LEDGER
WORK?
15
Users initiate
transactions
using their Digital
Signatures
Users Broadcast
their
transactions to
Nodes
One or more
Nodes begin
validating each
transaction
Nodes aggregate
validated
transactions into
Blocks
Nodes Broadcast
Blocks to each
other
Consensus
protocol used
Block reflecting
“true state” is
chained to prior
Block

WHERE MIGHT BLOCKCHAIN USE
CRYPTOGRAPHY?
16
• Digital Signatures
• Private/Public Keys
Initiation and Broadcasting
ofT
ransaction
• Proof ofWork and certain alternatives
V
alidation ofT
ransaction
• Hash Function
Chaining Blocks

THE POWER OF DISTRIBUTED LEDGERS
17
It can be used to allow
owners of assets to
exercise certain rights
associated with
ownership,and to
record the exercise of
those rights.
•Proxy Voting
It can be used to
record those
transfers of value or
ownership of assets
•These records may be
very difficult to alter,
such that they are
sometimes called
effectively immutable
It can be used to
transfer value or the
ownership of assets
•A human being or a
Smart Contract can
initiate the transfer
It can be used to
create value or issue
assets
It can be used without a central
authority by individuals or
entities with no basis to trust
each other
Thedegreeoftrustbetweenusersdeterminesthetechnological
configuration of a distributedledger
.

Blockchain Applications
Blockchain companies have begun disrupting all industries from financial services to online gaming, supply
chain and media. 2019 and 2020 will see the maturation of general purpose platforms that will give rise to use
cases across industries around the globe.
Use Cases
• Asset management
• Supply chain logistics & management
• Computing & storage
• Currency exchanges
• Media & adtech
• Online gaming
• Social media
• Lending & crowdfunding
• Healthcare data
• Insurance & underwriting
• Payments & banking
• Financial services
• Infrastructure
• Public records
Blockchain and Governance
• Transparent elections
• Ends identity theft
• Automation of land registries and title
transfers
• Criminal records
• Management of trial evidence
• Auditable universal basic income
• Transparent ownership of public companies
• Replacement of taxes with monetary policy
18

Case Study: ID PASS
ID PASS is a sustainable, open-sourced, blockchain-based digital identity solution. It enables governments and
humanitarian organizations to issue and verify a decentralized, private, trusted and recoverable form of identity
to all population subsets, including citizens, residents and refugees.
• Without a verifiable identity, one-seventh of the global population lacks the right to access essential
services such as citizenship, education, justice, health care, banking and insurance.
• However, having an identity document can also constitute a threat to the individual’s privacy and safety.
• What is needed is a form of identification that is trusted, secure, and allows the individual to choose when
to share or withhold personal information.
• This is facilitated by operating on a decentralized blockchain platform.
• The blockchain is used to anchor the identity while biometric data and other personally identifiable
information are stored off-chain, on a smart card.
ID PASS focuses on people who do not have a smartphone or internet access and provides them with a
verifiable identity by using blockchain, smart card technology and biometrics. Once enrolled, individuals have
the ability to access services using an identity that they can trust, that is verifiable, private, transportable and
under their control. ID PASS works both offline and online.
For more information, visit https://idpass.org
*Block.one does not endorse any third party or its products or services, even if they are mentioned herein. Block.one is not responsible for any linked content.
19

Case Study: Eva
Eva is a cooperative ride-sharing application built on the EOSIO blockchain protocol currently operating in
Canada with plans to launch globally in other cities.
• Raphaël Gaudreault, Eva’s co-founder and CTO, and Dardan Isufi, Eva’s Chief Optimization Officer
determined that traditional ride-sharing “companies capture around a quarter of market share versus
standard taxis, but the money is invested in capitalization in other countries. They simply aggregate the
economy and delocalize it.”
• Eva intends to address this issue by working as a network of cooperatives based around a new relationship
between driver and passenger members that removes the middleman.
• In the Eva model, 85% of each transaction goes to the driver, with the remainder being split between
members of the cooperative, an ecosystem “treasury” and a foundation responsible for maintaining the
network.
• Blockchain automates these allocations, and, according to Eva’s white paper, “improves the ability to
integrate different modes of payment while ensuring an anonymous, immutable, and transparent technical
protocol.”
• “Eva deploys an immutable contract based on the EOS blockchain to offer an open ledger for mobility that
keeps members’ data anonymous,” adds Raphaël Gaudreault.
For more information, visit http://eva.coop
20

Case Study: GeneOS
GeneOS is a blockchain-enabled data ownership, marketplace, and secure computing platform for genomic big
data. It is an all-in-one analytics platform that aggregates everything from DNA to sleep patterns to offer a
holistic view of user’s wellness.
• By using blockchain, all user data is anonymized and private
• In addition to offering a full wellness data kit, existing data from other external sources can be integrated
into GeneOS
• GeneOS takes the data and produces an aggregate health score, displays health findings relevant to the
user’s DNA and gives personalized health recommendations
• User genomic data is digitized into an income-producing asset - it can be rented out locally using privacy-
preserving technology for research and other commercial purposes resulting in the user receiving proceeds
in their accounts similar to dividend payments
For more information, visit https://geneos.me/
21

ADDITIONAL RESOURCES
• Bitcoin White Paper – Satoshi Nakamoto
• Blockchain Demo – Anders Brownworth
• Videos
• Blockchain for Business - An Introduction to Hyperledger Technologies -
edX.org
• Ethereum White Paper
• Guardtime – Blockchain like official site
• Hyperledger official site - Linux Foundation
• IBM Blockchain for Business – IBM Dev Center
• IBM Blockchain Essentials Course – IBM Dev Center
• IBM Blockchain Foundation Developer – IBM Dev Center
• Many more and pages are always changing
22

CENTRALIZED DB SYSTEMS
Software:
24
Application
SQL Front End
Query Processor
Transaction Proc.
File Access
P
M ...
• Simplifications:
 single front end
 one place to keep data, locks
 if processor fails, system fails, ...

HOMOGENEOUS DISTRIBUTED
DATABASES
• In a homogeneous distributed database
• All sites have identical software
• Are aware of each other and agree to cooperate in processing user
requests.
• Each site surrenders part of its autonomy in terms of right to change
schemas or software
• Appears to user as a single system
• In a heterogeneous distributed database
• Different sites may use different schemas and software
• Difference in schema is a major problem for query processing
• Difference in software is a major problem for transaction
processing
• Sites may not be aware of each other and may provide only
limited facilities for cooperation in transaction processing
25

DB ARCHITECTURES
(1) Shared memory
26
P P P
...
M

DB ARCHITECTURES
(2) Shared disk
27
...
...
P
M
P P
M M

DB ARCHITECTURES
(3) Shared nothing
28
P
M
P
M
P
M
...

DB ARCHITECTURES
(4) Hybrid example – Hierarchical or Clustered
29
M
P P P
...
M
P P P
...

• Typically, distributed DBs:
• Geographically distributed
• Data sharing is goal (may run into
heterogeneity, autonomy)
• Disconnected operation possible
30

DISTRIBUTED DATABASE
CHALLENGES
• Distributed Database Design
• Deciding what data goes where
• Depends on data access patterns of major applications
• Two subproblems:
• Fragmentation: partition tables into fragments
• Allocation: allocate fragments to nodes
31

DISTRIBUTED DATA STORAGE
32
• Assume relational data model
• Replication
• System maintains multiple copies of data, stored in
different sites, for faster retrieval and fault tolerance.
• Fragmentation
• Relation is partitioned into several fragments stored in
distinct sites
• Replication and fragmentation can be combined
• Relation is partitioned into several fragments: system
maintains several identical replicas of each such
fragment.

HORIZONTAL FRAGMENTATION OF ACCOUNT RELATION
33
branch_name account_number balance
Hillside
Hillside
Hillside
A-305
A-226
A-155
500
336
62
account1 = branch_name=“Hillside” (account )
branch_name account_number balance
Valleyview
Valleyview
Valleyview
Valleyview
A-177
A-402
A-408
A-639
205
10000
1123
750
account2 = branch_name=“Valleyview” (account )

VERTICAL FRAGMENTATION OF EMPLOYEE_INFO
RELATION
34 branch_name customer_name tuple_id
Hillside
Hillside
Valleyview
Valleyview
Hillside
Valleyview
Valleyview
Lowman
Camp
Camp
Kahn
Kahn
Kahn
Green
deposit1 = branch_name, customer_name, tuple_id (employee_info )
1
2
3
4
5
6
7
account_number balance tuple_id
500
336
205
10000
62
1123
750
1
2
3
4
5
6
7
A-305
A-226
A-177
A-402
A-155
A-408
A-639
deposit2 = account_number, balance, tuple_id (employee_info )

A DEFINITION
• Byzantine (www.m-w.com):
1: of, relating to, or characteristic of the ancient city of Byzantium
…
4b: intricately involved : labyrinthine <rules of Byzantine complexity>
• Lamport’s reason:
“I have long felt that, because it was posed as a cute problem about philosophers seated
around a table, Dijkstra's dining philosopher's problem received much more attention
than it deserves.”
(http://research.microsoft.com/users/lamport/pubs/pubs.html#byz)
35

BYZANTINE GENERALS PROBLEM
• Concerned with (binary) atomic broadcast
• All correct nodes receive same value
• If broadcaster correct, correct nodes receive broadcasted value
• Can use broadcast to build consensus protocols (aka, agreement)
• Consensus: think Byzantine fault-tolerant (BFT) Paxos
36

Synchronous Asynchronous
Fail-stop Byzantine
SYNCHRONOUS, BYZANTINE WORLD
37

FIRST PROTOCOL: NO CRYPTO
• Secure point-to-point links, but no crypto allowed
• Protocol OM(m): Recursive, exponential, all-to-all
• [Try to sketch protocol – see page 388]
• May be inefficient, but shows 3f+1 bound is tight
• [Discuss: Understand that this is for synchronous setup without crypto!]
• Need at least 3f+1 to tolerate f faulty!
• See figures 1 and 2
• How to fix? Signatures (for example). Or hash commitments, one-time
signatures, etc.
38

SECOND PROTOCOL: WITH CRYPTO
• Protocol SM(m)
• [Page 391, but can skip protocol]
• Given signatures, do m rounds of signing what you think was said. Many
messages (don’t need as many in absence of faults).
• Shows possible for any # of faults tolerated
• [Discuss. Understand: Synchronous, lots of messages, but possible.]
• [Skip odd topologies. Note that “signature” can be
emulated for random (not malicious) faults.]
39

BYZANTINE GENERALS PROBLEM (BGP)
• Goals
• Consensus (same plan) btw. loyal generals
• A small number of traitors cannot cause the loyals to adopt a bad plan
• Do not have to identify the traitors
40
• N Generals
• Some are traitors
• Message passing
A.C. 330
100K
50K
40K
30K
10K 20K
(commander)

BGP IN DISTRIBUTED SYSTEMS
• Goals
• All correct nodes share the same global info.
• Ensure that N corrupted nodes can not change the shared global info.,
and maximize N
• Identification of corrupted nodes would be needed
• What’s difference btw. BGP and consensus algo.?
• Fail-stop vs. fail-silent violation. Design goal.
41
• N Computers
• Some misbehave
• HW Fault,
SW bug,
Security attack,
misconfiguration
• Message passing
A thousand years later…

NAÏVE SOL. & 3-GENERAL IMPOSSIBILITY
• Naïve solution
• Each general sends its value, v(i), to all others
• Majority vote using v(1), v(2), …, v(n)
42
• Is it true that no solutions with fewer than
3m+1 generals can cope with m traitors? If so,
why?

3M-GENERAL IMPOSSIBILITY
43
– If there is a solution for 3m generals with m traitors,
it can be reduced to a solution of 3-General problem
“3m+1<=n” “3m+1>n”

• n = 4, m = 1
• L1 and L2 both receive v,v,x. (Consensus)
• L1 and L2 obey C
• All lieutenants receive x,y,z
• Lieutenant can identify commander
is a traitor
• What is communication complexity of this algorithm?
• Formal definition of OM(M)
– Command broadcasts its value to all lieutenants
– Each lieutenant acts as commander of OM(m-1)
SOLUTION I – ORAL MESSAGES
44

SOLUTION II – SIGNED MESSAGES
• Can we cope with any number of traitors? If so, how?
45
• Prevent traitors lie about the commander’s order
• Message are signed by commander
• The sign can be verified by all loyal lieutenants
• When lieutenant receives no new messages,
and select majority as the desired action
• All loyals receive the same set of cmds eventually
• If the commander is loyal, it works
• What if the commander
is not loyal?

PRACTICAL USE CASE OF BGP
46
• Distributed file systems
– Many small, latency-sensitive requests (tampering with files, lost
updates)
• Overlay multicast
– Transfers large volume of data (tampering with content,
freeloading)
• P2P email
– Complex, large, decentralized (Denial of service by misrouting)
 Not only consensus but also identifying faulty nodes is
important!

 Providing accountability for distributed systems
 Stores all I/O events as a log
 Selected nodes are responsible for auditing
the log
 Assumptions:
 System is modeled as deterministic state
machines
 State machines have reference
implementations
 Eventual communication
 Signe d message
PEERREVIEW
47
47

Module B
FAULT DETECTION
• How to recognize faults
in a log?
• Assumption
• Node can be modeled as
a deterministic state
machine
• To audit a node
• Start from a snapshot in
the log
• Replay inputs to a
trusted copy of the state
machine
• Check outputs against
the log
48
Module A
Module B
=?
Log
Network
Input
Output
State
machine
if ≠
Module A

COMMUNICATION ALGORITHRM
• All nodes keep a log of
their inputs & outputs
• Including all messages
• Each node has a set of
witnesses, who audit its
log periodically
• If the witnesses detect
misbehavior, they
• generate evidence
• make the evidence avai-
lable to other nodes
• Other nodes check evi-
dence, report fault
49
A's log
B's log
A
B
C
D
E
A's witnesses

TAMPER-PROOFING
50
A B
Message
Send(X)
Recv(Y)
Send(Z)
Recv(M)
H0
H1
H2
H3
H4
B's log
ACK
What if a node modifies its log
entries ?
Log entries form a hash chain
Inspired by secure histories
[Maniatis02]
Signed hash is included with
every message
 mi = (si, ti, ci)
hi = H(hi-1||si||ti||H(ci))
Commitement protocol
 Sender and recevier
commit to its current state
Hash(log)
Hash(log)

PROVABLE GUARANTEES
1) Completeness: Faults will be detected
2) Accuracy: Good nodes cannot be accused
51
If node commits a fault and has a correct witness,
then witness obtains
a proof of misbehavior (PoM), or
a challenge that the faulty node cannot answer
If node is correct
there can never be a PoM, and
it can answer any challenge

COMMUNICATION OVERHEAD
52
Baseline 1 2 3 4 5
100
80
60
40
20
0
Avg
traffic
(Kbps/node)
Number of witnesses
Baseline traffic
Signatures
and ACKs
Checking logs

MOTIVATION
BYZANTINE FAULT TOLERANCE
• Why we need BFT systems?
• Software systems : Valuable + Not reliable enough
• Amazon S3 crashed for hours in 2008
Reason: One corrupted bit
• Akami central nodes
• Hardware : Cheaper now
• Idea
• Use more hardware
Make software systems more reliable
53

BACKGROUND:
PRACTICAL BYZANTINE FAULT TOLERANCE
PBFT: establish order before execution
54
Client
Primary
Replica
Replica
Faulty
Replica
Pre-Prepare Prepare Commit Reply
Req, # n Req, # n?
OK, Req,
# n!
What is the
problem?
Before execution
4 network delays
Many messages

CLIENT CAN CORRECT ORDER
CASE 1
55
Client’s
Power
Client
Primary
Replica
Replica
Faulty
Replica
Pre-Prepare Spec-exe Reply
Order
Correct
To
This
state!
Order
Correct
Now!
Just do
it !
Just do
it !

CASE 2
56
Client’s
Power
Client
Primary
Replica
Replica
Faulty
Replica
Just do
it !
Just do
it !
Restart
Req!

CASE 3
57
Client’s
Power
Client
Primary
Replica
Replica
Replica
Just do
it !
Just do
it !
Change
Primary!

ZERO-KNOWLEDGE PROOFS
• Zero Knowledge Proof (ZKP) is an encryption scheme originally
proposed by MIT researchers Shafi Goldwasser, Silvio Micali and
Charles Rackoff in the 1980s.
• Zero-knowledge protocols are probabilistic assessments, which means
they don’t prove something with as much certainty as simply revealing
the entire information would.
• They provide unlinkable information that can together show the validity
of the assertion is probable.
• Currently, a website takes the user password as an input and then
compares its hash to the stored hash.
• Similarly a bank requires your credit score to provide you the loan
leaving your privacy and information leak risk at the mercy of the host
servers.
• If ZKP can be utilized, the client’s password is unknown the to verifier
and the login can still be authenticated. Before ZKP, we always
questioned the legitimacy of the prover or the soundness of the proof
system, but ZKP questions the morality of the verifier. What if the
verifier tries to leak the information?
59

PROPERTIES OF ZERO KNOWLEDGE PROOF
• Zero-Knowledge –
If the statement is true, the verifier will not know that the statement or was. Here
statement can be an absolute value or an algorithm.
• Completeness –
If the statement is true then an honest verifier can be convinced eventually.
• Soundness –
If the prover is dishonest, they can’t convince the verifier of the soundness of the
proof.
60

TYPES OF ZERO KNOWLEDGE PROOF :
• Interactive Zero Knowledge Proof –
It requires the verifier to constantly ask a series of questions about the “knowledge”
the prover possess. The above example of finding Waldo is interactive since the
“prover” did a series of actions to prove the about the soundness of the knowledge to
the verifier.
• Non-Interactive Zero Knowledge Proof –
For “interactive” solution to work, both the verifier and the prover needed to be online
at the same time making it difficult to scale up on the real world application. Non-
interactive Zero-Knowledge Proof do not require an interactive process, avoiding the
possibility of collusion. It requires picking a hash function to randomly pick the
challenge by the verifier. In 1986, Fiat and Shamir invented the Fiat-Shamir heuristic
and successfully changed the interactive zero-knowledge proof to non-interactive
zero knowledge proof.
61

EXAMPLES OF ZERO KNOWLEDGE
• Example-1: A Colour-blind friend and Two balls :
There are two friends Sachin and Sanchita, out of whom Sanchita is colour blind.
• Sachin has two balls and he needs to prove that both the balls our of different colour.
• Sanchita switches the balls randomly behind her back and shows it to Sachin who has to tell if
the balls are switched or not.
• If the balls are of the same colour and Sachin had given false information, the probability of him
answering correctly is 50%.
• When the activity is repeated several times, the probability of Sachin giving the correct answer
with the false information is significantly low. Here Sachin is the “prover” and Sanchita is the
“verifier”. Colour is the absolute information or the algorithm to be executed, and it is proved of
its soundness without revealing the information that is the colour to the verifier.
62

EXAMPLES OF ZERO KNOWLEDGE
• Example-2: Finding Waldo :
Finding Waldo is a game where you have to find a person called Waldo from a
snapshot of a huge crowd taken from above.
• Sachin has an algorithm to find Waldo but he doesn’t want to reveal it to Sanchita.
Sanchita wants to buy the algorithm but would need to check if the algorithm is
working.
• Sachin cuts a small hole on a cardboard and places over Waldo. Sachin is the
“prover” and Sanchita is the “verifier”. The algorithm is proved with zero
knowledge about it.
63

ZERO-KNOWLEDGE PROOF
Two parties: Prover P (PPT) and Verifier V (PPT)
(P is given witness for claim e.g., )
• Completeness: If claim is true honest prover can always convince honest verifier to accept.
• Soundness: If claim is false then Verifier should reject with probability at least ½. (Even if the
prover tries to cheat)
• Zero-Knowledge: Verifier doesn’t learn anything about prover’s input from the protocol (other
than that the claim is true).
• Formalizing this last statement is tricky
• Zero-Knowledge: should hold even if the attacker is dishonest!
66

Trans(1n,V’,P,x,w,rp,rv) transcript produced when V’ and P interact
• V’ is given input X (the problem instance e.g., 𝑋 = 𝑔𝑥
)
• P is given input X and w (a witness for the claim e.g., w=x)
• V’ and P use randomness rp and rv respectively
• Security parameter is n e.g., for encryption schemes, commitment schemes etc…
𝑿𝒏 = Trans(1n,V’,P,x,w) is a distribution over transcripts (over the randomness rp,rv)
(Blackbox Zero-Knowledge): There is a PPT simulator 𝑆 such that for every V’
(possibly cheating) S, with oracle access to V’, can simulate 𝑋𝑛 without a witness w.
Formally,
𝑋𝑛 𝑛∈ℕ ≡𝐶 𝑆𝑉′(.) 𝑥, 1𝑛
𝑛∈ℕ
67

Trans(1n,V’,P,x,w,rp,rv) transcript produced when V’ and P interact
• V’ is given input x (the problem instance e.g., 𝐴 = 𝑔𝑥1, B = 𝑔𝑥2 and 𝑧𝑏 )
• P is given input x and w (a witness for the claim e.g., x1 and x2)
• V’ and P’ use randomness rp and rw respectively
• Security parameter is n e.g., for encryption schemes, commitment schemes etc…
𝑿𝒏 = Trans(1n,V’,P’,x,w) is a distribution over transcripts (over the randomness rp,rw)
(Blackbox Zero-Knowledge): There is a PPT simulator 𝑆 such that for every V’
(possibly cheating) S, with oracle access to V’, can simulate 𝑋𝑛 without a witness w.
Formally,
𝑋𝑛 𝑛∈ℕ ≡𝐶 𝑆𝑉′(.) 𝑥, 1𝑛
𝑛∈ℕ
68
Simulator S is not
given witness w
Oracle V’(x,trans) will output
the next message V’ would
output given current transcript
trans

MEMORY HARD FUNCTION (MHF)
• Intuition: computation costs dominated by memory costs
vs.
• Data Independent Memory Hard Function (iMHF)
• Memory access pattern should not depend on input
69

BASIC FEATURES: HDFS
• Highly fault-tolerant
• High throughput
• Suitable for applications with large data sets
• Streaming access to file system data
• Can be built out of commodity hardware
9/11/2022
71

GOALS OF HDFS
• Very Large Distributed File System
• 10K nodes, 100 million files, 10PB
• Assumes Commodity Hardware
• Files are replicated to handle hardware failure
• Detect failures and recover from them
• Optimized for Batch Processing
• Data locations exposed so that computations can move to where
data resides
• Provides very high aggregate bandwidth

DISTRIBUTED FILE SYSTEM
• Single Namespace for entire cluster
• Data Coherency
• Write-once-read-many access model
• Client can only append to existing files
• Files are broken up into blocks
• Typically 64MB block size
• Each block replicated on multiple DataNodes
• Intelligent Client
• Client can find location of blocks
• Client accesses data directly from DataNode

Cat
Bat
Dog
Other
Words
(size:
TByte)
map
map
map
map
split
split
split
split
combine
combine
combine
reduce
reduce
reduce
part0
part1
part2
MAPREDUCE
9/11/2022
74

NAMENODE AND DATANODES
 Master/slave architecture
 HDFS cluster consists of a single Namenode, a master server that manages
the file system namespace and regulates access to files by clients.
 There are a number of DataNodes usually one per node in a cluster.
 The DataNodes manage storage attached to the nodes that they run on.
 HDFS exposes a file system namespace and allows user data to be stored
in files.
 A file is split into one or more blocks and set of blocks are stored in
DataNodes.
 DataNodes: serves read, write requests, performs block creation, deletion,
and replication upon instruction from Namenode.
9/11/2022
75

HDFS ARCHITECTURE
9/11/2022
76
Namenode
B
replication
Rack1 Rack2
Client
Blocks
Datanodes Datanodes
Client
Write
Read
Metadata ops
Metadata(Name, replicas..)
(/home/foo/data,6. ..
Block ops

FILE SYSTEM NAMESPACE
• Hierarchical file system with directories and files
• Create, remove, move, rename etc.
• Namenode maintains the file system
• Any meta information changes to the file system recorded by the
Namenode.
• An application can specify the number of replicas of the file needed:
replication factor of the file. This information is stored in the
Namenode.
9/11/2022
77

DATA REPLICATION
 HDFS is designed to store very large files across machines in a large
cluster.
 Each file is a sequence of blocks.
 All blocks in the file except the last are of the same size.
 Blocks are replicated for fault tolerance.
 Block size and replicas are configurable per file.
 The Namenode receives a Heartbeat and a BlockReport from each
DataNode in the cluster.
 BlockReport contains all the blocks on a Datanode.
9/11/2022
78

NAMENODE
 Keeps image of entire file system namespace and file Blockmap in
memory.
 4GB of local RAM is sufficient to support the above data structures that
represent the huge number of files and directories.
 When the Namenode starts up it gets the FsImage and Editlog from its
local file system, update FsImage with EditLog information and then
stores a copy of the FsImage on the filesytstem as a checkpoint.
 Periodic checkpointing is done. So that the system can recover back to
the last checkpointed state in case of a crash.
9/11/2022
79

DATANODE
 A Datanode stores data in files in its local file system.
 Datanode has no knowledge about HDFS filesystem
 It stores each block of HDFS data in a separate file.
 Datanode does not create all files in the same directory.
 It uses heuristics to determine optimal number of files per directory
and creates directories appropriately:
 Research issue?
 When the filesystem starts up it generates a list of all HDFS blocks and
send this report to Namenode: Blockreport.
9/11/2022
80

THE COMMUNICATION PROTOCOL
 All HDFS communication protocols are layered on top of the TCP/IP
protocol
 A client establishes a connection to a configurable TCP port on the
Namenode machine. It talks ClientProtocol with the Namenode.
 The Datanodes talk to the Namenode using Datanode protocol.
 RPC abstraction wraps both ClientProtocol and Datanode protocol.
 Namenode is simply a server and never initiates a request; it only
responds to RPC requests issued by DataNodes or clients.
9/11/2022
81

SPACE RECLAMATION
• When a file is deleted by a client, HDFS renames file to a file in be the
/trash directory for a configurable amount of time.
• A client can request for an undelete in this allowed time.
• After the specified time the file is deleted and the space is reclaimed.
• When the replication factor is reduced, the Namenode selects excess
replicas that can be deleted.
• Next heartbeat(?) transfers this information to the Datanode that clears
the blocks for use.
9/11/2022
82

DISTRIBUTED HASH TABLES – CURRENT STATE
• Active area of research for over 2 years now
• Ongoing work at almost every major university and lab.
• over 20 DHT proposals; as many for DHT applications
• IRIS : DHT-based, robust infrastructure for Internet-scale systems. 5
year, $12M, NSF-funded project
• Large, and growing, research community
• theoreticians, networks and systems researchers

WHAT IS A DHT?
• Hash Table
• data structure that maps “keys” to “values”
• essential building block in software systems
• Distributed Hash Table (DHT)
• similar, but spread across many hosts
• Interface
• insert(key, value)
• lookup(key)

HOW DO DHTS WORK?
Every DHT node supports a single operation:
• Given key as input; route messages to node
holding key
• DHTs are content-addressable

K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
DHT: BASIC IDEA

K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
DHT: BASIC IDEA
Neighboring nodes are “connected” at the application-level

K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
DHT: BASIC IDEA
Operation: take key as input; route messages to node holding key

K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
DHT: BASIC IDEA
insert(K1,V1)

insert(K1,V1)
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
DHT: BASIC IDEA

(K1,V1)
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
DHT: BASIC IDEA

retrieve (K1)
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
K V
DHT: BASIC IDEA

HOW TO DESIGN A DHT?
• State Assignment:
• what “(key, value) tables” does a node store?
• Network Topology:
• how does a node select its neighbors?
• Routing Algorithm:
• which neighbor to pick while routing to a destination?
• Various DHT algorithms make different choices
• CAN, Chord, Pastry, Tapestry, Plaxton, Viceroy, Kademlia, Skipnet, Symphony, Koorde, Apocrypha, Land,
ORDI …

STATE ASSIGNMENT IN CHORD DHT
• Nodes are randomly
chosen points on a
clock-wise Ring of
values
• Each node stores the id
space (values) between
itself and its
predecessor
d(100, 111) = 3
000
101
100
011
010
001
110
111

CHORD TOPOLOGY AND ROUTE
SELECTION
• Neighbor selection:
ith neighbor at 2i
distance
• Route selection: pick
neighbor closest to
destination
000
101
100
011
010
001
110
111 d(000, 001) = 1
d(000, 010) = 2
d(000, 001) = 4
110

1
Key space is a virtual d-dimensional Cartesian space
State Assignment in CAN

1 2

1
2
3

1
2
3
4

STATE ASSIGNMENT IN CAN

CAN TOPOLOGY AND ROUTE SELECTION
(a,b)
S
Route by forwarding to the neighbor “closest” to the destination

STATE AND NEIGHBOR ASSIGNMENT
IN PASTRY DHT
• Nodes are leaves in a tree
• logN neighbors in sub-trees of varying heights
001
000 011
010 101
100 111
110
h = 2
h = 1
h = 3

ROUTING IN PASTRY DHT
001
000 011
010 101
100 111
110
111
h = 3
h = 2
• Route to the sub-tree with the destination

ASIC-RESISTANT?
• «One-CPU-one-vote»
• Memory-hard puzzles
• Memory-bound
puzzles
• Scrypt – although
resitant, what
happened?
• DASH(x11)
• Changing/Moving
puzzles
• ASIC-resistant coins, as the name suggests, are cryptocurrencies with
ASIC-resistant algorithms.
• Their ecosystem is built in a way that disables users to mine coins with
ASIC machines.
• Therefore, mining these cryptocurrencies with ASICs is nearly impossible
• While some networks create ASIC-resistant coins to preserve and
increase the degree of decentralization of their blockchains, others do it
to make mining affordable for everyone.
• Ethereum is a popular example of an ASIC-resistant blockchain
KECCAK-256 HASHING ALGORITHM

ONE-WAY HASH FUNCTION
• Secret value is added before the hash and removed before
transmission.
106

SECURE HASH FUNCTIONS
• Purpose of the HASH function is to produce a
”fingerprint.
• Properties of a HASH function H :
1. H can be applied to a block of data at any size
2. H produces a fixed length output
3. H(x) is easy to compute for any given x.
4. For any given block x, it is computationally infeasible
to find x such that H(x) = h
5. For any given block x, it is computationally infeasible
to find with H(y) = H(x).
6. It is computationally infeasible to find any pair (x, y)
such that H(x) = H(y)
107
x
y 

SIMPLE HASH FUNCTION
• One-bit circular shift on the hash value after
each block is processed would improve
Henric Johnson
108

MESSAGE DIGEST GENERATION USING SHA-1
109

OTHER SECURE HASH FUNCTIONS
110
SHA-1 MD5 RIPEMD-160
Digest length 160 bits 128 bits 160 bits
Basic unit of
processing
512 bits 512 bits 512 bits
Number of steps 80 (4 rounds of 20) 64 (4 rounds of 16) 160 (5 paired
rounds of 16)
Maximum
message size
264-1 bits
 

APPLICATIONS FOR PUBLIC-KEY CRYPTOSYSTEMS
• Three categories:
• Encryption/decryption: The sender encrypts a message with the recipient’s public
key.
• Digital signature: The sender ”signs” a message with its private key.
• Key echange: Two sides cooperate two exhange a session key.
111

HISTORY OF DIGITAL SIGNATURE ALGORITHM
(DSA)
• 1982: the U.S government solicited proposals for a public key signature standard
• 1984: ElGamal Signatures [ElGamal84]
• 1990: Schnorr Signatures: various improvements [Schnorr90], U.S. Patent 4,995,082
• 1991: NIST proposes DSS=DSA. U.S. Patent 5,231,668 by David W. Kravitz (NSA).
• 1992: Panel discussion at EUROCRYPT 1992: trapdoor in DSS? [DLLMORS92]
• 1992: Public comments [RHAL92, …]
• 1992: NIST publishes “Response to Comments on the NIST Proposed Digital Signature Standard”, CRYPTO 1992 [SK92]
• 1994: DSS standard FIPS 186. Includes 1024-bit moduli. First digital signature recognized by any government
• 1992: Vanstone proposes EC variant of DSA
• 1995: IEEE P1363 working proposes current form of ECDSA
• 1998: ISO 14888-3: ECDSA
• 1999: ANSI X9.62: ECDSA
• 2000: IEEE 1363-200: ECDSA
• 2000: FIPS 186-2 includes ECDSA: 15 elliptic curves, chosen by Jerry Solinas (NSA), including NIST P-256
• 2019: FIPS 186-5 (draft) forbids signing with DSA (verify still ok), includes EdDSA
Collected from: [Bernstein14], Wikipedia, various

ECDSA TODAY
• SSL: 20% ECDSA, 80% RSA
(https://notary.icsi.berkeley.edu/, July 2018)
• TLS: 25% ECDSA, 75% RSA
(https://telemetry.mozilla.org/, May 2021)
• 99% support ecdsa_secp256r1_sha256
(https://tlsfingerprint.io/sig-algs, May 2021)
• Certificates: 7% ECDSA
(https://censys.io/certificates, May 2021)
• Car2X communication (IEEE 1609-2): 100% ECDSA?
• Mainstream cryptocurrencies: 100% ECDSA
• Threshold signature schemes [CGGM20,CLST21,
YCX21, …]
• Multi-party signing protocols [Lindell17, …]
• Adaptor signatures
• DSA: ???
kiltz@home % openssl version
LibreSSL 2.8.3
kiltz@home % openssl s_client –connect
wikipedia.org:443
CONNECTED(00000005)
[…]
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-ECDSA-AES256-GCM-SHA384
[…]
*thanks to Nadia Henniger, Dan Brown, Juraj Somorovsky, Robert Merget, Tim Güneysu, Peter Schwabe

GENERIC DSA: GENDSA[𝔾, H, F ][BROWN02]
Gen:
x ← ℤp; X = gx
pk = X; sk = x
Return (pk, sk)
Sign(sk,m):
r ← ℤp
*; R = gr
t = f(R); h = H(m)
s = (h + t∙x) / r mod p
Return 𝜎 = (s,t) ∊ ℤp×ℤp
Ver(pk, m, 𝜎 = (s,t)):
h = H(m)
R’ = (gh∙Xt)1/s
t’ = f(R’)
Return 1 iff t = t’
1. (𝔾, ∙) = group of prime order p, <g>=𝔾
2. H: {0,1}* → ℤp hash function
(SHA)
3. f: 𝔾 → ℤp conversion function
• DSA: 𝔾 = subgroup of 𝔽q, f(R) := R mod p
• ECDSA: 𝔾 = EC over 𝔽, f(R) := Rx mod p
x-coordinate of R=(Rx,Ry) ∊𝔽×𝔽
• Special case of ElGamal signatures [ElGamal84]
• Deterministic ECDSA (RFC 6979): r ← ℤp r := H(sk,m)
• Country-specific variants
• Russian (EC)GOST (RFC 7091)
• Chinese SM2 (ISO/IEC 11889:2015)
• German (EC)GDSA (ISO/IEC 15946-2)
+
set
containment
checks
for
s,
t,
R'

SECURITY
• UF-CMA: UnForgeability against Chosen Message Attack
pk
mi (1 ≤ i ≤ q)
𝜎i
m*, 𝜎*
UF-NMA: UnForgeability against No Message Attack (aka Key Only Attack)
• No signing queries allowed
UF-1CMA: UnForgeability against 1-per-message Chosen Message Attack
• Signing queries mi distinct
• Equivalent to UF-NMA for deterministic signing
m* ≠ mi
s Strong
(m*, 𝜎*) ≠ (mi,𝜎i)

PROVABLE SECURITY
• Provable security = “security of A implies security of B” (A ⇒ B)
Reduction
A B
• Crypto can be secure in practice, without being provably secure
• Crypto can be insecure in practice, even being provably secure
• Talk will not cover SCA: ECDSA is very prone to SCA!
“Every natural implementation of ECDSA makes heavy use of secret branches and secret array
indices” [Bernstein14]
• Talk will not cover “bad randomness” (Playstation 3 Hack 2010) or “randomness leakage”

Blockchain Technology Basics Course

Recommended

Recommended

More Related Content

Similar to Blockchain Technology Basics Course

Similar to Blockchain Technology Basics Course (20)

Recently uploaded

Recently uploaded (20)

Blockchain Technology Basics Course