Downloaded 10 times
Module: Mutable Content in IPFS
- 1.
- 2. a CONTENT ADDRESSING CONTENTDISCOVERY & ROUTING CONTENT EXCHANGE ● Chunking ● Linking Chunks in Merkle DAGs ● From Data to Data Structures with IPLD ● Anatomy of the IPFS CID ● Routing & Provider Records ● DHT-based Routing ● Gossip-based Routing ● Bitswap ● GraphSync MUTABLE NAMES & MESSAGE DELIVERY ● Dynamic Data ● IPNS ● PubSub ● CRDTs IPFS Components
- 3. Agenda ➔ Motivation ➔ MessagingLayer ◆ IPFS/libp2p PubSub ◆ PubSub Routers (Floodsub & Gossipsub) ➔ Mutable Pointers ◆ Self-Certified Names (IPNS) ◆ IPNS over DHT, PubSub & DNS ➔ Type Systems for Distributed Applications ◆ Conflict Free Replicated Data Types (CRDT) ➔ Access Controls for Distributed Applications ◆ Cryptographic ACL / Capabilities System ➔ Examples of tools and libraries available in the ecosystem
- 4. Motivation Motivation Messaging Layer Mutable Pointers TypeSystems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
- 5. Enable Web 2.0like Rich Experiences in the Web 3.0
- 6.
- 7. ➔ IPFS givesus immutable identifiers (CIDs), however, Rich Web Apps need dynamic content that changes rapidly ➔ Ability to create, edit and update existing data, from text to images, videos and so on. ➔ Propagating updates to other participants, created a shared view into the latest state of a dataset ➔ Grant and revoke read and write permissions to data ➔ Support multiple interaction patterns, from one-to-many (e.g. Blog), many-to-many (e.g. Social Networks) and many-to-one (e.g. RSS/feed subscriptions) What is Mutable Content about?
- 8. Needed to supportmultiple classes of applications
- 9. OSI Layer System ApplicationLayer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer
- 10. OSI Layer System libp2p ➔Process Addressing ➔ End-to-End Connections ➔ End-to-End Encryption ➔ Reliability Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer
- 11. Messaging Layer Motivation Messaging Layer MutablePointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
- 12. Benefits ➔ Layer fordistributing updates ➔ Message Oriented Comms -> Simple to program ➔ Support for different topologies -> Different types of interaction patterns supported ➔ Loose Coupling -> Separation of Concerns ➔ Scalable -> Adapt as your network grows A Messaging Layer supported by P2P PubSub is key for supporting different interaction patterns Challenges ➔ Permission-less network -> can’t control who join/leaves ➔ Network topology is bottom-up (e.g. brokerless) ➔ Network Churn ➔ Optimizations for Latency/Bandwidth/Delivery Guarantees are about tradeoffs, not one-size fits all
- 13. ➔ Topic basedinterface ➔ Design Goals ◆ Reliability: All messages get delivered to all peers subscribed to the topic. ◆ Speed: Messages are delivered quickly. ◆ Efficiency: The network is not flooded with excess copies of messages. ◆ Resilience: Peers can join and leave the network without disrupting it. There is no central point of failure. ◆ Scale: Topics can have enormous numbers of subscribers and handle a large throughput of messages. ◆ Simplicity: The system is simple to understand and implement. Each peer only needs to remember a small amount of state. IPFS/libp2p PubSub ➔ Parameterizable ➔ Two main implementations ◆ FloodSub ◆ Gossipsub
- 14. ➔ Design ◆ Simplestpossible design ◆ Ambient Peer Discovery (e.g. IPFS Main Network’s DHT, MulticastDNS and other systems) ◆ Routing is achieved by Flooding ➔ Pros ◆ Robust delivery guarantees, even in the presence of high network churn ◆ Minimum latency delivery ➔ Cons ◆ Huge bandwidth overhead (duplicate messages received) ◆ Unbounded degree flooding (wasterful) FloodSub Time
- 15. FloodSub ➔ Simulation ◆ 100peers ◆ 5 msg/s ◆ Run for 2s ◆ Time expansion 10x for visualisation
- 16. ➔ Bleeding edgeP2P PubSub ➔ Design ◆ Self Stabilizing algorithm ◆ 2 Networks: Metadata (control plane) & Message (data plane) ◆ Nodes establish local meshes with reciprocal agreements on subscriptions ◆ Degree bounded by default to 6 ◆ Nodes send Gossip with ● GRAFT & PRUNE to establish peering agreements ● Gossip about what messages have been delivered and seen ◆ Eager push/Lazy pull model ◆ Scoring function to reward benevolent peers and penalize bad behaving peers ◆ Support for protocol extensions ➔ Pros ◆ Resilient to Churn ◆ Resilient to Sybil, Eclipse & Spam Attacks ◆ Minimizes Bandwidth usage ➔ Cons ◆ For small networks, it doesn’t achieve the same minimum latency guarantees as Floodsub Gossipsub with hardening extensions
- 17. ➔ Simulation ◆ 100peers ◆ 5 msg/s ◆ Run for 2s ◆ Time expansion 10x for visualisation Gossipsub with hardening extensions
- 18. ➔ Evaluation Performedwith over 10 000 peers ➔ Demonstrated successfully how Gossipsub is resilient to a variety of attacks ➔ The implementation of Gossipsub and the code used for the evaluation is all Open Source ➔ Paper Publish with evaluation and Architecture Evaluation Report & Paper
- 19. ➔ Documents allknown challenges & existing solutions ➔ Invitation for collaboration in solving the challenges at new lengths Open Problem Statement
- 20. libp2p Application Layer Presentation Layer SessionLayer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map (OSI Layer System)
- 21. Mutable Pointers Motivation Messaging Layer MutablePointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
- 22. Goals ➔ No centralcentral coordinator to distribute updates ➔ Ability to certify that the updates originate from the author Decentralized updates are a requirement for several use cases Challenges ➔ Different expectations of delivery depending on the application ➔ Scalability (# of nodes) and frequency (# of updates)
- 23. ➔ Self-Certified Naming ➔Design ◆ Verifiability: ◆ Versatility: A name can point to one or more immutable addresses ◆ Transport independent: Not be tied to a specific method of distribution. ➔ Names (aka Pointers) are referenced by the PublicKey of the signer vs. the Content of the name IPNS, the InterPlanetary NameSystem ➔ Several implementations: ◆ IPNS over DHT ◆ IPNS over PubSub ◆ IPNS over DNS ◆ IPNS over ETH Domains ◆ IPNS over Namecoin (one of the first implementations) ◆ IPNS over QRCodes? NFC? BLE Beacons? Multiple possibilities
- 24. ➔ Each nameis a Public/Private Key Pair ➔ The Private Key is used to signed statements (aka records) containing the state of the name ➔ The Public Key is used by clients for ◆ Discovering the new records published ◆ Certify the record update was indeed issued by the author ➔ Every time the owner of the name wants to issue an update, it simply creates a new record, signs it and broadcasts it over one of the selected transports (e.g DHT, PubSub, DNS, etc) ➔ Human Readable naming is enabled via a registrar. Examples: ◆ DNS some-domain.com -> TXT Record /ipns/Qm_CID of Public Key.. -> /ipfs/Qm_CID of content ◆ UnstoppableDomains some-domain.crypto -> /ipns/Qm_CID of Public Key.. -> /ipfs/Qm_CID of content IPNS Architecture
- 25. ➔ Documents allknown challenges & existing solutions ➔ Invitation for collaboration in solving the challenges at new lengths Open Problem Statements
- 26. libp2p Application Layer Presentation Layer SessionLayer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map (OSI Layer System) IPNS
- 27. ➔ “A distributedsystem is one in which the failure of a computer you didn't even know existed can render your own computer unusable.” Leslie Lamport ➔ Synchronization errors happen ◆ Simultaneous Writers ◆ Propagation Delay ➔ Distributed Consensus is not a trivial problem ➔ There must be a better way! 💡 Mutable Pointers are not enough for Distributed Applications
- 28. Type Systems for DistributedApps Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
- 29. Goals ➔ No needfor central coordinator and avoid central points of failure and/or chokepoints (e.g. case for Operational Transforms) ➔ Support multiple times of collaboration Convergent state and/or Consensus over Distributed Data Challenges ➔ Avoid resource intensive and slow constructions (e.g. PBFT, Nakamoto Consensus, etc) ➔ Support Web scale ➔ Fast sync, Snapshotting and Garbage Collection
- 30. ➔ Fairly newand large field of distributed systems research ➔ Design ◆ Merge function is defined a priority and distributed to all participants ◆ Independent of the order of the events received, all participants will know how to merge the events and converge on the same state ➔ CRDTs are not a one-size fits all data structure, instead they are many data structures that fit specific use cases: ◆ G-Set (Grow-only Set) ◆ 2P-Set (Two-Phase Set) ◆ LWW-Element-Set (Last-Write-Wins-Element-Set) ◆ OR-Set (Observed-Remove Set) ◆ Sequence/Ordered Set ◆ Many more.. Conflict-Free Replicated Data Types (CRDTs) ➔ Pros ◆ Order of reception is not important, as long as the events arrive ◆ Does not require any live interactivity with all participants ◆ Can be used to build all sorts of applications, from collaborative doc editors, chats and much more ➔ Cons ◆ Can generate a lot of non-trivially garbage collectable objects for long running programs
- 31. libp2p Application Layer Presentation Layer SessionLayer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map (OSI Layer System) IPNS CRDT
- 32. Access Controls for DistributedApps Motivation Messaging Layer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
- 33. Goals ➔ Grant andRemove access to data ➔ No need for central coordinator and avoid central points of failure and/or chokepoints (e.g. case for Operational Transforms) ➔ Support multiple times of collaboration Access Controls Challenges ➔ Avoid resource intensive and slow constructions (e.g. PBFT, Nakamoto Consensus, etc) to agree on the ACL ➔ Support Web scale ➔ Fast sync, Snapshotting and Garbage Collection
- 34. ➔ Private collaborationsover data in a Public Network ➔ Design ◆ Symmetric Keys give Read Access ● Only the ones with the key can decrypt and therefore read ◆ Asymmetric Keys give Write Access ● Similar to how IPNS gives ownership of who is trusted to updated the Record ➔ Pros ◆ Fine grain control of Read & Write Access ◆ Authorship is verifiable ◆ Does not require a centralized party for coordination ➔ Cons ◆ Require nodes to have a way to synchronize on the latest state of ACL to ensure that only nodes allowed receive the latest updates Cryptographic ACLs aka Capabilities Systems
- 35. libp2p Application Layer Presentation Layer SessionLayer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map OSI Layer System onto DWeb Stack IPNS CRDT Cryptographic ACL
- 36. libp2p Application Layer Presentation Layer SessionLayer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Map OSI Layer System onto DWeb Stack IPNS CRDT Cryptographic ACL
- 37. ➔ Open SourceCollaborative Text Editor ➔ IPNS to get the latest release the App ➔ IPFS to load the contents ➔ Public Key Pair & Symmetric Key at the URL ➔ CRDT to create the shared state ➔ PubSub to distribute messages ➔ Features ◆ Real-time collaboration ◆ Conflict-Free PeerPad as an example of the vertical integration
- 38. Available Tools & Libraries Motivation MessagingLayer Mutable Pointers Type Systems for Distributed Apps Access Controls for Distributed Apps Mutable Content Tools & Libraries
- 39. libp2p Application Layer Presentation Layer SessionLayer Transport Layer Network Layer Data Link Layer Physical Layer PubSub Open Source Libraries & Tools Available IPNS CRDT Cryptographic ACL
- 40. Thank you forwatching Reach out in case you have questions or comments! resnetlab@protocol.ai https://github.com/protocol/ResNetLab/discussions