SlideShare a Scribd company logo
1 of 40
Download to read offline
Module:
Mutable Content
ResNetLab on Tour
a
CONTENT ADDRESSING CONTENT DISCOVERY
& ROUTING
CONTENT EXCHANGE
● Chunking
● Linking Chunks
in Merkle DAGs
● From Data to
Data Structures with IPLD
● Anatomy of the IPFS CID
● Routing & Provider
Records
● DHT-based Routing
● Gossip-based Routing
● Bitswap
● GraphSync
MUTABLE NAMES &
MESSAGE DELIVERY
● Dynamic Data
● IPNS
● PubSub
● CRDTs
IPFS Components
Agenda
➔ Motivation
➔ Messaging Layer
◆ IPFS/libp2p PubSub
◆ PubSub Routers (Floodsub & Gossipsub)
➔ Mutable Pointers
◆ Self-Certified Names (IPNS)
◆ IPNS over DHT, PubSub & DNS
➔ Type Systems for Distributed Applications
◆ Conflict Free Replicated Data Types (CRDT)
➔ Access Controls for Distributed Applications
◆ Cryptographic ACL / Capabilities System
➔ Examples of tools and libraries available in the ecosystem
Motivation
Motivation
Messaging Layer
Mutable Pointers
Type Systems for
Distributed Apps
Access Controls for
Distributed Apps
Mutable Content
Tools & Libraries
Enable Web 2.0 like
Rich Experiences
in the Web 3.0
Without mandating
centralized infrastructure
➔ IPFS gives us immutable identifiers (CIDs), however, Rich Web Apps need dynamic
content that changes rapidly
➔ Ability to create, edit and update existing data, from text to images, videos and so on.
➔ Propagating updates to other participants, created a shared view into the latest state
of a dataset
➔ Grant and revoke read and write permissions to data
➔ Support multiple interaction patterns, from one-to-many (e.g. Blog), many-to-many (e.g.
Social Networks) and many-to-one (e.g. RSS/feed subscriptions)
What is Mutable Content about?
Needed to support multiple classes of applications
OSI Layer System
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
OSI Layer System
libp2p
➔ Process Addressing
➔ End-to-End Connections
➔ End-to-End Encryption
➔ Reliability
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
Messaging Layer
Motivation
Messaging Layer
Mutable Pointers
Type Systems for
Distributed Apps
Access Controls for
Distributed Apps
Mutable Content
Tools & Libraries
Benefits
➔ Layer for distributing updates
➔ Message Oriented Comms -> Simple to program
➔ Support for different topologies -> Different types
of interaction patterns supported
➔ Loose Coupling -> Separation of Concerns
➔ Scalable -> Adapt as your network grows
A Messaging Layer supported by
P2P PubSub is key for supporting
different interaction patterns
Challenges
➔ Permission-less network -> can’t control who
join/leaves
➔ Network topology is bottom-up (e.g. brokerless)
➔ Network Churn
➔ Optimizations for Latency/Bandwidth/Delivery
Guarantees are about tradeoffs, not one-size fits all
➔ Topic based interface
➔ Design Goals
◆ Reliability: All messages get delivered to all peers
subscribed to the topic.
◆ Speed: Messages are delivered quickly.
◆ Efficiency: The network is not flooded with excess
copies of messages.
◆ Resilience: Peers can join and leave the network
without disrupting it. There is no central point of
failure.
◆ Scale: Topics can have enormous numbers of
subscribers and handle a large throughput of
messages.
◆ Simplicity: The system is simple to understand and
implement. Each peer only needs to remember a
small amount of state.
IPFS/libp2p PubSub
➔ Parameterizable
➔ Two main implementations
◆ FloodSub
◆ Gossipsub
➔ Design
◆ Simplest possible design
◆ Ambient Peer Discovery (e.g. IPFS Main Network’s DHT, MulticastDNS and
other systems)
◆ Routing is achieved by Flooding
➔ Pros
◆ Robust delivery guarantees, even in the presence of high
network churn
◆ Minimum latency delivery
➔ Cons
◆ Huge bandwidth overhead
(duplicate messages received)
◆ Unbounded degree flooding
(wasterful)
FloodSub
Time
FloodSub
➔ Simulation
◆ 100 peers
◆ 5 msg/s
◆ Run for 2s
◆ Time expansion 10x
for visualisation
➔ Bleeding edge P2P PubSub
➔ Design
◆ Self Stabilizing algorithm
◆ 2 Networks: Metadata (control plane) & Message (data plane)
◆ Nodes establish local meshes with reciprocal agreements on subscriptions
◆ Degree bounded by default to 6
◆ Nodes send Gossip with
● GRAFT & PRUNE to establish peering agreements
● Gossip about what messages have been delivered and seen
◆ Eager push/Lazy pull model
◆ Scoring function to reward benevolent peers and penalize bad behaving peers
◆ Support for protocol extensions
➔ Pros
◆ Resilient to Churn
◆ Resilient to Sybil, Eclipse & Spam Attacks
◆ Minimizes Bandwidth usage
➔ Cons
◆ For small networks, it doesn’t achieve the same minimum latency guarantees as
Floodsub
Gossipsub with
hardening extensions
➔ Simulation
◆ 100 peers
◆ 5 msg/s
◆ Run for 2s
◆ Time expansion 10x
for visualisation
Gossipsub with
hardening extensions
➔ Evaluation Performed with over 10 000 peers
➔ Demonstrated successfully how Gossipsub is resilient to a variety of attacks
➔ The implementation of Gossipsub and the code used for the evaluation is all Open Source
➔ Paper Publish with evaluation and Architecture
Evaluation Report & Paper
➔ Documents all known challenges & existing solutions
➔ Invitation for collaboration in solving the challenges at new lengths
Open Problem Statement
libp2p
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
PubSub
Map (OSI Layer System)
Mutable Pointers
Motivation
Messaging Layer
Mutable Pointers
Type Systems for
Distributed Apps
Access Controls for
Distributed Apps
Mutable Content
Tools & Libraries
Goals
➔ No central central coordinator to distribute
updates
➔ Ability to certify that the updates originate
from the author
Decentralized updates are a
requirement for several use cases
Challenges
➔ Different expectations of delivery depending on
the application
➔ Scalability (# of nodes) and frequency
(# of updates)
➔ Self-Certified Naming
➔ Design
◆ Verifiability:
◆ Versatility: A name can point to one or more
immutable addresses
◆ Transport independent: Not be tied to a specific
method of distribution.
➔ Names (aka Pointers) are referenced by the
PublicKey of the signer vs. the Content of the
name
IPNS, the
InterPlanetary NameSystem
➔ Several implementations:
◆ IPNS over DHT
◆ IPNS over PubSub
◆ IPNS over DNS
◆ IPNS over ETH Domains
◆ IPNS over Namecoin (one of the first
implementations)
◆ IPNS over QRCodes? NFC? BLE Beacons?
Multiple possibilities
➔ Each name is a Public/Private Key Pair
➔ The Private Key is used to signed statements (aka records)
containing the state of the name
➔ The Public Key is used by clients for
◆ Discovering the new records published
◆ Certify the record update was indeed issued by the author
➔ Every time the owner of the name wants to issue an update, it
simply creates a new record, signs it and broadcasts it over
one of the selected transports (e.g DHT, PubSub, DNS, etc)
➔ Human Readable naming is enabled via a registrar. Examples:
◆ DNS some-domain.com -> TXT Record /ipns/Qm_CID of
Public Key.. -> /ipfs/Qm_CID of content
◆ UnstoppableDomains some-domain.crypto -> /ipns/Qm_CID
of Public Key.. -> /ipfs/Qm_CID of content
IPNS Architecture
➔ Documents all known challenges & existing solutions
➔ Invitation for collaboration in solving the challenges at new lengths
Open Problem Statements
libp2p
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
PubSub
Map (OSI Layer System)
IPNS
➔ “A distributed system is one in which the failure of a computer you
didn't even know existed can render your own computer unusable.”
Leslie Lamport
➔ Synchronization errors happen
◆ Simultaneous Writers
◆ Propagation Delay
➔ Distributed Consensus is not a trivial problem
➔ There must be a better way! 💡
Mutable Pointers are not enough
for Distributed Applications
Type Systems for
Distributed Apps
Motivation
Messaging Layer
Mutable Pointers
Type Systems for
Distributed Apps
Access Controls for
Distributed Apps
Mutable Content
Tools & Libraries
Goals
➔ No need for central coordinator and avoid
central points of failure and/or chokepoints
(e.g. case for Operational Transforms)
➔ Support multiple times of collaboration
Convergent state and/or
Consensus over Distributed Data
Challenges
➔ Avoid resource intensive and slow constructions
(e.g. PBFT, Nakamoto Consensus, etc)
➔ Support Web scale
➔ Fast sync, Snapshotting and Garbage Collection
➔ Fairly new and large field of distributed
systems research
➔ Design
◆ Merge function is defined a priority and
distributed to all participants
◆ Independent of the order of the events received,
all participants will know how to merge the
events and converge on the same state
➔ CRDTs are not a one-size fits all data
structure, instead they are many data
structures that fit specific use cases:
◆ G-Set (Grow-only Set)
◆ 2P-Set (Two-Phase Set)
◆ LWW-Element-Set (Last-Write-Wins-Element-Set)
◆ OR-Set (Observed-Remove Set)
◆ Sequence/Ordered Set
◆ Many more..
Conflict-Free Replicated Data
Types (CRDTs)
➔ Pros
◆ Order of reception is not important, as long as the
events arrive
◆ Does not require any live interactivity with all
participants
◆ Can be used to build all sorts of applications, from
collaborative doc editors, chats and much more
➔ Cons
◆ Can generate a lot of non-trivially garbage collectable
objects for long running programs
libp2p
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
PubSub
Map (OSI Layer System)
IPNS
CRDT
Access Controls for
Distributed Apps
Motivation
Messaging Layer
Mutable Pointers
Type Systems for
Distributed Apps
Access Controls for
Distributed Apps
Mutable Content
Tools & Libraries
Goals
➔ Grant and Remove access to data
➔ No need for central coordinator and avoid
central points of failure and/or chokepoints
(e.g. case for Operational Transforms)
➔ Support multiple times of collaboration
Access Controls
Challenges
➔ Avoid resource intensive and slow constructions
(e.g. PBFT, Nakamoto Consensus, etc) to agree
on the ACL
➔ Support Web scale
➔ Fast sync, Snapshotting and Garbage Collection
➔ Private collaborations over data in a Public Network
➔ Design
◆ Symmetric Keys give Read Access
● Only the ones with the key can decrypt and therefore
read
◆ Asymmetric Keys give Write Access
● Similar to how IPNS gives ownership of who is trusted to
updated the Record
➔ Pros
◆ Fine grain control of Read & Write Access
◆ Authorship is verifiable
◆ Does not require a centralized party for coordination
➔ Cons
◆ Require nodes to have a way to synchronize on the latest
state of ACL to ensure that only nodes allowed receive the
latest updates
Cryptographic ACLs
aka Capabilities Systems
libp2p
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
PubSub
Map OSI Layer System onto DWeb Stack
IPNS
CRDT Cryptographic ACL
libp2p
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
PubSub
Map OSI Layer System onto DWeb Stack
IPNS
CRDT Cryptographic ACL
➔ Open Source Collaborative Text Editor
➔ IPNS to get the latest release the App
➔ IPFS to load the contents
➔ Public Key Pair & Symmetric Key at the URL
➔ CRDT to create the shared state
➔ PubSub to distribute messages
➔ Features
◆ Real-time collaboration
◆ Conflict-Free
PeerPad as an example of the
vertical integration
Available Tools &
Libraries
Motivation
Messaging Layer
Mutable Pointers
Type Systems for
Distributed Apps
Access Controls for
Distributed Apps
Mutable Content
Tools & Libraries
libp2p
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
PubSub
Open Source Libraries & Tools Available
IPNS
CRDT Cryptographic ACL
Thank you for watching
Reach out in case you have questions or comments!
resnetlab@protocol.ai https://github.com/protocol/ResNetLab/discussions

More Related Content

What's hot

Mind Map of Big Data Technologies and Concepts
Mind Map of Big Data Technologies and ConceptsMind Map of Big Data Technologies and Concepts
Mind Map of Big Data Technologies and ConceptsAmir Hadad
 
Scaling Hadoop at LinkedIn
Scaling Hadoop at LinkedInScaling Hadoop at LinkedIn
Scaling Hadoop at LinkedInDataWorks Summit
 
NATS vs HTTP for Interservice Communication
NATS vs HTTP for Interservice CommunicationNATS vs HTTP for Interservice Communication
NATS vs HTTP for Interservice CommunicationNATS
 
Mining a Large Web Corpus
Mining a Large Web CorpusMining a Large Web Corpus
Mining a Large Web CorpusRobert Meusel
 
Hudi architecture, fundamentals and capabilities
Hudi architecture, fundamentals and capabilitiesHudi architecture, fundamentals and capabilities
Hudi architecture, fundamentals and capabilitiesNishith Agarwal
 
Apache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data ProcessingApache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data ProcessingDataWorks Summit
 
Introdução à Neo4j
Introdução à Neo4j Introdução à Neo4j
Introdução à Neo4j Neo4j
 
Apache Arrow Flight Overview
Apache Arrow Flight OverviewApache Arrow Flight Overview
Apache Arrow Flight OverviewJacques Nadeau
 
Introduction to apache spark
Introduction to apache spark Introduction to apache spark
Introduction to apache spark Aakashdata
 
FHIR for clinicians
FHIR for clinicians FHIR for clinicians
FHIR for clinicians David Hay
 
Apache Sentry for Hadoop security
Apache Sentry for Hadoop securityApache Sentry for Hadoop security
Apache Sentry for Hadoop securitybigdatagurus_meetup
 
Introduction to Apache Hadoop Eco-System
Introduction to Apache Hadoop Eco-SystemIntroduction to Apache Hadoop Eco-System
Introduction to Apache Hadoop Eco-SystemMd. Hasan Basri (Angel)
 
Introduction to NoSQL Databases
Introduction to NoSQL DatabasesIntroduction to NoSQL Databases
Introduction to NoSQL DatabasesDerek Stainer
 
Big Data Architecture and Design Patterns
Big Data Architecture and Design PatternsBig Data Architecture and Design Patterns
Big Data Architecture and Design PatternsJohn Yeung
 
Building large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudiBuilding large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudiBill Liu
 
RedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter CailliauRedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter CailliauRedis Labs
 
Build Mandarin AI Conversational Agent with Rasa
Build Mandarin AI Conversational Agent with RasaBuild Mandarin AI Conversational Agent with Rasa
Build Mandarin AI Conversational Agent with RasaHao-Yuan Chen
 
Introduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse ArchitectureIntroduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse ArchitectureDatabricks
 
Pig Tutorial | Twitter Case Study | Apache Pig Script and Commands | Edureka
Pig Tutorial | Twitter Case Study | Apache Pig Script and Commands | EdurekaPig Tutorial | Twitter Case Study | Apache Pig Script and Commands | Edureka
Pig Tutorial | Twitter Case Study | Apache Pig Script and Commands | EdurekaEdureka!
 

What's hot (20)

Mind Map of Big Data Technologies and Concepts
Mind Map of Big Data Technologies and ConceptsMind Map of Big Data Technologies and Concepts
Mind Map of Big Data Technologies and Concepts
 
Scaling Hadoop at LinkedIn
Scaling Hadoop at LinkedInScaling Hadoop at LinkedIn
Scaling Hadoop at LinkedIn
 
NATS vs HTTP for Interservice Communication
NATS vs HTTP for Interservice CommunicationNATS vs HTTP for Interservice Communication
NATS vs HTTP for Interservice Communication
 
Mining a Large Web Corpus
Mining a Large Web CorpusMining a Large Web Corpus
Mining a Large Web Corpus
 
Hudi architecture, fundamentals and capabilities
Hudi architecture, fundamentals and capabilitiesHudi architecture, fundamentals and capabilities
Hudi architecture, fundamentals and capabilities
 
Apache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data ProcessingApache Tez - A New Chapter in Hadoop Data Processing
Apache Tez - A New Chapter in Hadoop Data Processing
 
Introdução à Neo4j
Introdução à Neo4j Introdução à Neo4j
Introdução à Neo4j
 
Apache Arrow Flight Overview
Apache Arrow Flight OverviewApache Arrow Flight Overview
Apache Arrow Flight Overview
 
Introduction to apache spark
Introduction to apache spark Introduction to apache spark
Introduction to apache spark
 
FHIR for clinicians
FHIR for clinicians FHIR for clinicians
FHIR for clinicians
 
Apache Sentry for Hadoop security
Apache Sentry for Hadoop securityApache Sentry for Hadoop security
Apache Sentry for Hadoop security
 
Introduction to Apache Hadoop Eco-System
Introduction to Apache Hadoop Eco-SystemIntroduction to Apache Hadoop Eco-System
Introduction to Apache Hadoop Eco-System
 
Druid
DruidDruid
Druid
 
Introduction to NoSQL Databases
Introduction to NoSQL DatabasesIntroduction to NoSQL Databases
Introduction to NoSQL Databases
 
Big Data Architecture and Design Patterns
Big Data Architecture and Design PatternsBig Data Architecture and Design Patterns
Big Data Architecture and Design Patterns
 
Building large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudiBuilding large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudi
 
RedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter CailliauRedisGraph A Low Latency Graph DB: Pieter Cailliau
RedisGraph A Low Latency Graph DB: Pieter Cailliau
 
Build Mandarin AI Conversational Agent with Rasa
Build Mandarin AI Conversational Agent with RasaBuild Mandarin AI Conversational Agent with Rasa
Build Mandarin AI Conversational Agent with Rasa
 
Introduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse ArchitectureIntroduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse Architecture
 
Pig Tutorial | Twitter Case Study | Apache Pig Script and Commands | Edureka
Pig Tutorial | Twitter Case Study | Apache Pig Script and Commands | EdurekaPig Tutorial | Twitter Case Study | Apache Pig Script and Commands | Edureka
Pig Tutorial | Twitter Case Study | Apache Pig Script and Commands | Edureka
 

Similar to Module: Mutable Content in IPFS

Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Anton Nazaruk
 
Everything you always wanted to know about Distributed databases, at devoxx l...
Everything you always wanted to know about Distributed databases, at devoxx l...Everything you always wanted to know about Distributed databases, at devoxx l...
Everything you always wanted to know about Distributed databases, at devoxx l...javier ramirez
 
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre ZembBuilding a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre ZembStreamNative
 
The Open Source and Cloud Part of Oracle Big Data Cloud Service for Beginners
The Open Source and Cloud Part of Oracle Big Data Cloud Service for BeginnersThe Open Source and Cloud Part of Oracle Big Data Cloud Service for Beginners
The Open Source and Cloud Part of Oracle Big Data Cloud Service for BeginnersEdelweiss Kammermann
 
Glusterfs and openstack
Glusterfs  and openstackGlusterfs  and openstack
Glusterfs and openstackopenstackindia
 
Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...
Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...
Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...HostedbyConfluent
 
(Current22) Let's Monitor The Conditions at the Conference
(Current22) Let's Monitor The Conditions at the Conference(Current22) Let's Monitor The Conditions at the Conference
(Current22) Let's Monitor The Conditions at the ConferenceTimothy Spann
 
Music city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lakeMusic city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lakeTimothy Spann
 
Hail hydrate! from stream to lake using open source
Hail hydrate! from stream to lake using open sourceHail hydrate! from stream to lake using open source
Hail hydrate! from stream to lake using open sourceTimothy Spann
 
Captial One: Why Stream Data as Part of Data Transformation?
Captial One: Why Stream Data as Part of Data Transformation?Captial One: Why Stream Data as Part of Data Transformation?
Captial One: Why Stream Data as Part of Data Transformation?ScyllaDB
 
Pacemaker+DRBD
Pacemaker+DRBDPacemaker+DRBD
Pacemaker+DRBDDan Frincu
 
Apache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-AriApache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-AriDemi Ben-Ari
 
Big data conference europe real-time streaming in any and all clouds, hybri...
Big data conference europe   real-time streaming in any and all clouds, hybri...Big data conference europe   real-time streaming in any and all clouds, hybri...
Big data conference europe real-time streaming in any and all clouds, hybri...Timothy Spann
 
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATSDeploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATSNATS
 
Lec+3-Introduction-to-Distributed-Systems.pdf
Lec+3-Introduction-to-Distributed-Systems.pdfLec+3-Introduction-to-Distributed-Systems.pdf
Lec+3-Introduction-to-Distributed-Systems.pdfsamaghorab
 
HPC Controls Future
HPC Controls FutureHPC Controls Future
HPC Controls Futurercastain
 
Strimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUp
Strimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUpStrimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUp
Strimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUpJosé Román Martín Gil
 
Hadoop Ecosystem and Low Latency Streaming Architecture
Hadoop Ecosystem and Low Latency Streaming ArchitectureHadoop Ecosystem and Low Latency Streaming Architecture
Hadoop Ecosystem and Low Latency Streaming ArchitectureInSemble
 
ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...
ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...
ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...Docker-Hanoi
 

Similar to Module: Mutable Content in IPFS (20)

Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?Big Data Streams Architectures. Why? What? How?
Big Data Streams Architectures. Why? What? How?
 
Everything you always wanted to know about Distributed databases, at devoxx l...
Everything you always wanted to know about Distributed databases, at devoxx l...Everything you always wanted to know about Distributed databases, at devoxx l...
Everything you always wanted to know about Distributed databases, at devoxx l...
 
Cloud storage
Cloud storageCloud storage
Cloud storage
 
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre ZembBuilding a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre Zemb
 
The Open Source and Cloud Part of Oracle Big Data Cloud Service for Beginners
The Open Source and Cloud Part of Oracle Big Data Cloud Service for BeginnersThe Open Source and Cloud Part of Oracle Big Data Cloud Service for Beginners
The Open Source and Cloud Part of Oracle Big Data Cloud Service for Beginners
 
Glusterfs and openstack
Glusterfs  and openstackGlusterfs  and openstack
Glusterfs and openstack
 
Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...
Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...
Let’s Monitor Conditions at the Conference With Timothy Spann & David Kjerrum...
 
(Current22) Let's Monitor The Conditions at the Conference
(Current22) Let's Monitor The Conditions at the Conference(Current22) Let's Monitor The Conditions at the Conference
(Current22) Let's Monitor The Conditions at the Conference
 
Music city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lakeMusic city data Hail Hydrate! from stream to lake
Music city data Hail Hydrate! from stream to lake
 
Hail hydrate! from stream to lake using open source
Hail hydrate! from stream to lake using open sourceHail hydrate! from stream to lake using open source
Hail hydrate! from stream to lake using open source
 
Captial One: Why Stream Data as Part of Data Transformation?
Captial One: Why Stream Data as Part of Data Transformation?Captial One: Why Stream Data as Part of Data Transformation?
Captial One: Why Stream Data as Part of Data Transformation?
 
Pacemaker+DRBD
Pacemaker+DRBDPacemaker+DRBD
Pacemaker+DRBD
 
Apache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-AriApache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-Ari
 
Big data conference europe real-time streaming in any and all clouds, hybri...
Big data conference europe   real-time streaming in any and all clouds, hybri...Big data conference europe   real-time streaming in any and all clouds, hybri...
Big data conference europe real-time streaming in any and all clouds, hybri...
 
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATSDeploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATS
 
Lec+3-Introduction-to-Distributed-Systems.pdf
Lec+3-Introduction-to-Distributed-Systems.pdfLec+3-Introduction-to-Distributed-Systems.pdf
Lec+3-Introduction-to-Distributed-Systems.pdf
 
HPC Controls Future
HPC Controls FutureHPC Controls Future
HPC Controls Future
 
Strimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUp
Strimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUpStrimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUp
Strimzi - Where Apache Kafka meets OpenShift - OpenShift Spain MeetUp
 
Hadoop Ecosystem and Low Latency Streaming Architecture
Hadoop Ecosystem and Low Latency Streaming ArchitectureHadoop Ecosystem and Low Latency Streaming Architecture
Hadoop Ecosystem and Low Latency Streaming Architecture
 
ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...
ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...
ContainerDayVietnam2016: Kubernetes State-of-the-art Container Management Pla...
 

Recently uploaded

Q4-1-Illustrating-Hypothesis-Testing.pptx
Q4-1-Illustrating-Hypothesis-Testing.pptxQ4-1-Illustrating-Hypothesis-Testing.pptx
Q4-1-Illustrating-Hypothesis-Testing.pptxeditsforyah
 
Magic exist by Marta Loveguard - presentation.pptx
Magic exist by Marta Loveguard - presentation.pptxMagic exist by Marta Loveguard - presentation.pptx
Magic exist by Marta Loveguard - presentation.pptxMartaLoveguard
 
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书zdzoqco
 
定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一
定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一
定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一Fs
 
办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一
办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一
办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一z xss
 
Film cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasaFilm cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasa494f574xmv
 
Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170
Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170
Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170Sonam Pathan
 
A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)
A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)
A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)Christopher H Felton
 
Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作
Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作
Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作ys8omjxb
 
Top 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxTop 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxDyna Gilbert
 
Call Girls Near The Suryaa Hotel New Delhi 9873777170
Call Girls Near The Suryaa Hotel New Delhi 9873777170Call Girls Near The Suryaa Hotel New Delhi 9873777170
Call Girls Near The Suryaa Hotel New Delhi 9873777170Sonam Pathan
 
定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一
定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一
定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一Fs
 
NSX-T and Service Interfaces presentation
NSX-T and Service Interfaces presentationNSX-T and Service Interfaces presentation
NSX-T and Service Interfaces presentationMarko4394
 
定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一
定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一
定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一Fs
 
SCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is prediSCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is predieusebiomeyer
 
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书rnrncn29
 
Contact Rya Baby for Call Girls New Delhi
Contact Rya Baby for Call Girls New DelhiContact Rya Baby for Call Girls New Delhi
Contact Rya Baby for Call Girls New Delhimiss dipika
 
Blepharitis inflammation of eyelid symptoms cause everything included along w...
Blepharitis inflammation of eyelid symptoms cause everything included along w...Blepharitis inflammation of eyelid symptoms cause everything included along w...
Blepharitis inflammation of eyelid symptoms cause everything included along w...Excelmac1
 

Recently uploaded (20)

Q4-1-Illustrating-Hypothesis-Testing.pptx
Q4-1-Illustrating-Hypothesis-Testing.pptxQ4-1-Illustrating-Hypothesis-Testing.pptx
Q4-1-Illustrating-Hypothesis-Testing.pptx
 
Hot Sexy call girls in Rk Puram 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in  Rk Puram 🔝 9953056974 🔝 Delhi escort ServiceHot Sexy call girls in  Rk Puram 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Rk Puram 🔝 9953056974 🔝 Delhi escort Service
 
Magic exist by Marta Loveguard - presentation.pptx
Magic exist by Marta Loveguard - presentation.pptxMagic exist by Marta Loveguard - presentation.pptx
Magic exist by Marta Loveguard - presentation.pptx
 
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
 
定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一
定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一
定制(AUT毕业证书)新西兰奥克兰理工大学毕业证成绩单原版一比一
 
办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一
办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一
办理(UofR毕业证书)罗切斯特大学毕业证成绩单原版一比一
 
Film cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasaFilm cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasa
 
Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170
Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170
Call Girls In The Ocean Pearl Retreat Hotel New Delhi 9873777170
 
A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)
A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)
A Good Girl's Guide to Murder (A Good Girl's Guide to Murder, #1)
 
Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作
Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作
Potsdam FH学位证,波茨坦应用技术大学毕业证书1:1制作
 
Top 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxTop 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptx
 
Call Girls Near The Suryaa Hotel New Delhi 9873777170
Call Girls Near The Suryaa Hotel New Delhi 9873777170Call Girls Near The Suryaa Hotel New Delhi 9873777170
Call Girls Near The Suryaa Hotel New Delhi 9873777170
 
定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一
定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一
定制(Lincoln毕业证书)新西兰林肯大学毕业证成绩单原版一比一
 
NSX-T and Service Interfaces presentation
NSX-T and Service Interfaces presentationNSX-T and Service Interfaces presentation
NSX-T and Service Interfaces presentation
 
定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一
定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一
定制(Management毕业证书)新加坡管理大学毕业证成绩单原版一比一
 
SCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is prediSCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is predi
 
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
 
young call girls in Uttam Nagar🔝 9953056974 🔝 Delhi escort Service
young call girls in Uttam Nagar🔝 9953056974 🔝 Delhi escort Serviceyoung call girls in Uttam Nagar🔝 9953056974 🔝 Delhi escort Service
young call girls in Uttam Nagar🔝 9953056974 🔝 Delhi escort Service
 
Contact Rya Baby for Call Girls New Delhi
Contact Rya Baby for Call Girls New DelhiContact Rya Baby for Call Girls New Delhi
Contact Rya Baby for Call Girls New Delhi
 
Blepharitis inflammation of eyelid symptoms cause everything included along w...
Blepharitis inflammation of eyelid symptoms cause everything included along w...Blepharitis inflammation of eyelid symptoms cause everything included along w...
Blepharitis inflammation of eyelid symptoms cause everything included along w...
 

Module: Mutable Content in IPFS

  • 2. a CONTENT ADDRESSING CONTENT DISCOVERY & ROUTING CONTENT EXCHANGE ● Chunking ● Linking Chunks in Merkle DAGs ● From Data to Data Structures with IPLD ● Anatomy of the IPFS CID ● Routing & Provider Records ● DHT-based Routing ● Gossip-based Routing ● Bitswap ● GraphSync MUTABLE NAMES & MESSAGE DELIVERY ● Dynamic Data ● IPNS ● PubSub ● CRDTs IPFS Components
  • 3. Agenda ➔ Motivation ➔ Messaging Layer ◆ IPFS/libp2p PubSub ◆ PubSub Routers (Floodsub & Gossipsub) ➔ Mutable Pointers ◆ Self-Certified Names (IPNS) ◆ IPNS over DHT, PubSub & DNS ➔ Type Systems for Distributed Applications ◆ Conflict Free Replicated Data Types (CRDT) ➔ Access Controls for Distributed Applications ◆ Cryptographic ACL / Capabilities System ➔ Examples of tools and libraries available in the ecosystem
  • 4. Motivation Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
  • 5. Enable Web 2.0 like Rich Experiences in the Web 3.0
  • 7. ➔ IPFS gives us immutable identifiers (CIDs), however, Rich Web Apps need dynamic content that changes rapidly ➔ Ability to create, edit and update existing data, from text to images, videos and so on. ➔ Propagating updates to other participants, created a shared view into the latest state of a dataset ➔ Grant and revoke read and write permissions to data ➔ Support multiple interaction patterns, from one-to-many (e.g. Blog), many-to-many (e.g. Social Networks) and many-to-one (e.g. RSS/feed subscriptions) What is Mutable Content about?
  • 8. Needed to support multiple classes of applications
  • 9. OSI Layer System Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer
  • 10. OSI Layer System libp2p ➔ Process Addressing ➔ End-to-End Connections ➔ End-to-End Encryption ➔ Reliability Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer
  • 11. Messaging Layer Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
  • 12. Benefits ➔ Layer for distributing updates ➔ Message Oriented Comms -> Simple to program ➔ Support for different topologies -> Different types of interaction patterns supported ➔ Loose Coupling -> Separation of Concerns ➔ Scalable -> Adapt as your network grows A Messaging Layer supported by P2P PubSub is key for supporting different interaction patterns Challenges ➔ Permission-less network -> can’t control who join/leaves ➔ Network topology is bottom-up (e.g. brokerless) ➔ Network Churn ➔ Optimizations for Latency/Bandwidth/Delivery Guarantees are about tradeoffs, not one-size fits all
  • 13. ➔ Topic based interface ➔ Design Goals ◆ Reliability: All messages get delivered to all peers subscribed to the topic. ◆ Speed: Messages are delivered quickly. ◆ Efficiency: The network is not flooded with excess copies of messages. ◆ Resilience: Peers can join and leave the network without disrupting it. There is no central point of failure. ◆ Scale: Topics can have enormous numbers of subscribers and handle a large throughput of messages. ◆ Simplicity: The system is simple to understand and implement. Each peer only needs to remember a small amount of state. IPFS/libp2p PubSub ➔ Parameterizable ➔ Two main implementations ◆ FloodSub ◆ Gossipsub
  • 14. ➔ Design ◆ Simplest possible design ◆ Ambient Peer Discovery (e.g. IPFS Main Network’s DHT, MulticastDNS and other systems) ◆ Routing is achieved by Flooding ➔ Pros ◆ Robust delivery guarantees, even in the presence of high network churn ◆ Minimum latency delivery ➔ Cons ◆ Huge bandwidth overhead (duplicate messages received) ◆ Unbounded degree flooding (wasterful) FloodSub Time
  • 15. FloodSub ➔ Simulation ◆ 100 peers ◆ 5 msg/s ◆ Run for 2s ◆ Time expansion 10x for visualisation
  • 16. ➔ Bleeding edge P2P PubSub ➔ Design ◆ Self Stabilizing algorithm ◆ 2 Networks: Metadata (control plane) & Message (data plane) ◆ Nodes establish local meshes with reciprocal agreements on subscriptions ◆ Degree bounded by default to 6 ◆ Nodes send Gossip with ● GRAFT & PRUNE to establish peering agreements ● Gossip about what messages have been delivered and seen ◆ Eager push/Lazy pull model ◆ Scoring function to reward benevolent peers and penalize bad behaving peers ◆ Support for protocol extensions ➔ Pros ◆ Resilient to Churn ◆ Resilient to Sybil, Eclipse & Spam Attacks ◆ Minimizes Bandwidth usage ➔ Cons ◆ For small networks, it doesn’t achieve the same minimum latency guarantees as Floodsub Gossipsub with hardening extensions
  • 17. ➔ Simulation ◆ 100 peers ◆ 5 msg/s ◆ Run for 2s ◆ Time expansion 10x for visualisation Gossipsub with hardening extensions
  • 18. ➔ Evaluation Performed with over 10 000 peers ➔ Demonstrated successfully how Gossipsub is resilient to a variety of attacks ➔ The implementation of Gossipsub and the code used for the evaluation is all Open Source ➔ Paper Publish with evaluation and Architecture Evaluation Report & Paper
  • 19. ➔ Documents all known challenges & existing solutions ➔ Invitation for collaboration in solving the challenges at new lengths Open Problem Statement
  • 20. libp2p Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map (OSI Layer System)
  • 21. Mutable Pointers Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
  • 22. Goals ➔ No central central coordinator to distribute updates ➔ Ability to certify that the updates originate from the author Decentralized updates are a requirement for several use cases Challenges ➔ Different expectations of delivery depending on the application ➔ Scalability (# of nodes) and frequency (# of updates)
  • 23. ➔ Self-Certified Naming ➔ Design ◆ Verifiability: ◆ Versatility: A name can point to one or more immutable addresses ◆ Transport independent: Not be tied to a specific method of distribution. ➔ Names (aka Pointers) are referenced by the PublicKey of the signer vs. the Content of the name IPNS, the InterPlanetary NameSystem ➔ Several implementations: ◆ IPNS over DHT ◆ IPNS over PubSub ◆ IPNS over DNS ◆ IPNS over ETH Domains ◆ IPNS over Namecoin (one of the first implementations) ◆ IPNS over QRCodes? NFC? BLE Beacons? Multiple possibilities
  • 24. ➔ Each name is a Public/Private Key Pair ➔ The Private Key is used to signed statements (aka records) containing the state of the name ➔ The Public Key is used by clients for ◆ Discovering the new records published ◆ Certify the record update was indeed issued by the author ➔ Every time the owner of the name wants to issue an update, it simply creates a new record, signs it and broadcasts it over one of the selected transports (e.g DHT, PubSub, DNS, etc) ➔ Human Readable naming is enabled via a registrar. Examples: ◆ DNS some-domain.com -> TXT Record /ipns/Qm_CID of Public Key.. -> /ipfs/Qm_CID of content ◆ UnstoppableDomains some-domain.crypto -> /ipns/Qm_CID of Public Key.. -> /ipfs/Qm_CID of content IPNS Architecture
  • 25. ➔ Documents all known challenges & existing solutions ➔ Invitation for collaboration in solving the challenges at new lengths Open Problem Statements
  • 26. libp2p Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map (OSI Layer System) IPNS
  • 27. ➔ “A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.” Leslie Lamport ➔ Synchronization errors happen ◆ Simultaneous Writers ◆ Propagation Delay ➔ Distributed Consensus is not a trivial problem ➔ There must be a better way! 💡 Mutable Pointers are not enough for Distributed Applications
  • 28. Type Systems for Distributed Apps Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
  • 29. Goals ➔ No need for central coordinator and avoid central points of failure and/or chokepoints (e.g. case for Operational Transforms) ➔ Support multiple times of collaboration Convergent state and/or Consensus over Distributed Data Challenges ➔ Avoid resource intensive and slow constructions (e.g. PBFT, Nakamoto Consensus, etc) ➔ Support Web scale ➔ Fast sync, Snapshotting and Garbage Collection
  • 30. ➔ Fairly new and large field of distributed systems research ➔ Design ◆ Merge function is defined a priority and distributed to all participants ◆ Independent of the order of the events received, all participants will know how to merge the events and converge on the same state ➔ CRDTs are not a one-size fits all data structure, instead they are many data structures that fit specific use cases: ◆ G-Set (Grow-only Set) ◆ 2P-Set (Two-Phase Set) ◆ LWW-Element-Set (Last-Write-Wins-Element-Set) ◆ OR-Set (Observed-Remove Set) ◆ Sequence/Ordered Set ◆ Many more.. Conflict-Free Replicated Data Types (CRDTs) ➔ Pros ◆ Order of reception is not important, as long as the events arrive ◆ Does not require any live interactivity with all participants ◆ Can be used to build all sorts of applications, from collaborative doc editors, chats and much more ➔ Cons ◆ Can generate a lot of non-trivially garbage collectable objects for long running programs
  • 31. libp2p Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map (OSI Layer System) IPNS CRDT
  • 32. Access Controls for Distributed Apps Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
  • 33. Goals ➔ Grant and Remove access to data ➔ No need for central coordinator and avoid central points of failure and/or chokepoints (e.g. case for Operational Transforms) ➔ Support multiple times of collaboration Access Controls Challenges ➔ Avoid resource intensive and slow constructions (e.g. PBFT, Nakamoto Consensus, etc) to agree on the ACL ➔ Support Web scale ➔ Fast sync, Snapshotting and Garbage Collection
  • 34. ➔ Private collaborations over data in a Public Network ➔ Design ◆ Symmetric Keys give Read Access ● Only the ones with the key can decrypt and therefore read ◆ Asymmetric Keys give Write Access ● Similar to how IPNS gives ownership of who is trusted to updated the Record ➔ Pros ◆ Fine grain control of Read & Write Access ◆ Authorship is verifiable ◆ Does not require a centralized party for coordination ➔ Cons ◆ Require nodes to have a way to synchronize on the latest state of ACL to ensure that only nodes allowed receive the latest updates Cryptographic ACLs aka Capabilities Systems
  • 35. libp2p Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map OSI Layer System onto DWeb Stack IPNS CRDT Cryptographic ACL
  • 36. libp2p Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map OSI Layer System onto DWeb Stack IPNS CRDT Cryptographic ACL
  • 37. ➔ Open Source Collaborative Text Editor ➔ IPNS to get the latest release the App ➔ IPFS to load the contents ➔ Public Key Pair & Symmetric Key at the URL ➔ CRDT to create the shared state ➔ PubSub to distribute messages ➔ Features ◆ Real-time collaboration ◆ Conflict-Free PeerPad as an example of the vertical integration
  • 38. Available Tools & Libraries Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
  • 39. libp2p Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Open Source Libraries & Tools Available IPNS CRDT Cryptographic ACL
  • 40. Thank you for watching Reach out in case you have questions or comments! resnetlab@protocol.ai https://github.com/protocol/ResNetLab/discussions