Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

RDM#2- The Distributed Web

592 views

Published on

"The Distributed Web" A talk about building distributed web apps for the second Research & Development Meetup

Published in: Engineering

RDM#2- The Distributed Web

  1. 1. RDM#2 LISBON Research & Development Meetup
  2. 2. LISBON David Dias, P2P SE&R @ Protocol Labs João Ascensão, Data Scientist @ Uniplaces tw: @daviddias gh: @diasdavid tw: @jtascensao gh: @jtascensao
  3. 3. LISBON What is RDM?
  4. 4. LISBON Space to invest time in learning things that are hard to grasp
  5. 5. LISBON world of possibilities
  6. 6. LISBON what actually gets researched
  7. 7. LISBON what gets developed
  8. 8. LISBON what gets deployed
  9. 9. LISBON what most people are doing
  10. 10. PROTOCOL LABS
  11. 11. LISBON FORMAT - New topics / Research - Hacking + Digesting + Publishing
  12. 12. LISBON TODAY - The Distributed Web - Machine Learning / AI
  13. 13. The Distributed Web
  14. 14. Serve data to everywhere
  15. 15. • Motivations • IPFS, the Protocol • MerkleDAG (Directed Acyclic Graph) • DHT (Distributed Hash Table)
  16. 16. • Motivations • IPFS, the Protocol, MerkleDAG, IPNS, Network • IPFS in action • IPFS, the project • filecoin • libp2p • starship • gx • full IPFS JavaScript implementation • mutable file system and the files API • Other DHT • Structure P2P Networks • Unstructured P2P Networks • NAT Traversal • Relay/Turn • registry-mirror • smart chunking • Transports • Multiplexers • Self Describing Data Types • Search • Station • and more
  17. 17. Location Addressing
  18. 18. http://10.20.30.40/foo/bar/baz.png location path http://example.com/foo/bar/baz.png
  19. 19. http://10.20.30.40/foo/bar/baz.png you
  20. 20. http://10.20.30.40/foo/bar/baz.png 10.20.30.40 you
  21. 21. http://10.20.30.40/foo/bar/baz.png 10.20.30.40 you
  22. 22. 200 MB
  23. 23. 200 MB x 30 x 8 = 48 GB
  24. 24. 2,544,837,694
  25. 25. 2,387,990,609 508.968 PB
  26. 26. Bandwidth
  27. 27. https://www.backblaze.com/blog/price-gap-storage-vs-bandwidth/
  28. 28. Latency
  29. 29. Amazon Google DataCenters
  30. 30. Disconnected / Offline
  31. 31. low bw interference congestion travelisp outagedatacenter
  32. 32. Permanence Security IoT Control Offline Bandwidth Disconnected
  33. 33. InterPlanetary FileSystem
  34. 34. a protocol to upgrade the web
  35. 35. InterPlanetary FileSystem
  36. 36. InterPlanetary FileSystem
  37. 37. DISTRIBUTED PERMANENT MERKLE THE WEB
  38. 38. PermanentDistributed Safer SmarterOffline Faster
  39. 39. SFS web DHT+
  40. 40. web DHT SFS routing network exchange merkledag naming applications The Stack
  41. 41. routing network exchange merkledag naming applicationsThe Stack Defining the Data Moving the Data Using the Data IPLD libp2p IPNS
  42. 42. 8AyaTQMhJklMo Ga3WaDgwcAEPc KQeipgDu64qaj kr3fNcVt1LVMD KQsXSobGtbNhF sv3qA44E7iXAy gcuSfVhW8RV3z R9F3MFvbVaqf9
  43. 43. 8AyaTQMhJklMo Ga3WaDgwcAEPc KQeipgDu64qaj kr3fNcVt1LVMD hash( ) -> KQsXSobGtbNhF KQsXSobGtbNhF sv3qA44E7iXAy gcuSfVhW8RV3z R9F3MFvbVaqf9
  44. 44. 8AyaTQMhJklMo Ga3WaDgwcAEPc KQeipgDu64qaj kr3fNcVt1LVMD hash( ) -> KQsXSobGtbNhF KQsXSobGtbNhF sv3qA44E7iXAy gcuSfVhW8RV3z R9F3MFvbVaqf9 MERKLE LINK
  45. 45. cvs/svn
  46. 46. cvs/svn
  47. 47. cvs/svn
  48. 48. merkledag
  49. 49. in IPFS data forms a dag it’s called the merkle dag because the links are hashes
  50. 50. any data structures are represented as dags
  51. 51. unix files and dirs any data structures are represented as dags
  52. 52. blob, tree, commit unix files and dirs any data structures are represented as dags
  53. 53. blob, tree, commit unix files and dirs any data structures are represented as dags blocks, txns, wallets
  54. 54. blob, tree, commit unix files and dirs any data structures are represented as dags blocks, txns, wallets kv-stores
  55. 55. blob, tree, commit unix files and dirs any data structures are represented as dags blocks, txns, wallets kv-stores
  56. 56. unixfs as a dag
  57. 57. files are dag nodes unixfs as a dag
  58. 58. files are dag nodes BIG files may be split into many unixfs as a dag
  59. 59. directories are also dag nodes files are dag nodes BIG files may be split into many unixfs as a dag
  60. 60. which link to others directories are also dag nodes files are dag nodes BIG files may be split into many unixfs as a dag
  61. 61. http://10.20.30.40/foo/bar/baz.png location path http://example.com/foo/bar/baz.png
  62. 62. http://10.20.30.40/foo/bar/baz.png location path http://example.com/foo/bar/baz.png /ipns/example.com/foo/bar/baz.png /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png content path
  63. 63. 10.20.30.40 you HTTP http://10.20.30.40/foo/bar/baz.png /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
  64. 64. http://10.20.30.40/foo/bar/baz.png 10.20.30.40 you /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png IPFS
  65. 65. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png content path
  66. 66. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png content path
  67. 67. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png content path foo bar baz.png QmW98pJrc6FZ6
  68. 68. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png content path foo:HB HB HC HD bar:HC baz.png:HDQmW98pJrc6FZ6 foo bar baz.png
  69. 69. DHT
  70. 70. DHT are the Holy Grail* of P2P Networks *from a survey paper
  71. 71. 1 5 6 10 50 55
  72. 72. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500
  73. 73. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500
  74. 74. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500 hash( )
  75. 75. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500 hash( ) = 91
  76. 76. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500 hash( ) = 91
  77. 77. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500 Who has fim , which hash( ) = 91?
  78. 78. The Distributed Web
  79. 79. • Applications with no-origin • Leverage network cache • Resilient to network splits
  80. 80. Serve data to everywhere
  81. 81. Serve data from everywhere
  82. 82. David Dias - http://daviddias.me IPFS http://ipfs.io Protocol Labs http://ipn.io
  83. 83. Thank you! David Dias - http://daviddias.me IPFS http://ipfs.io Protocol Labs http://ipn.io

×