Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Datanet Website

147 views

Published on

Datanet CRDT based data synchonization system, a 4 minute introduction

Published in: Software
  • Be the first to comment

  • Be the first to like this

Datanet Website

  1. 1. Datanet Open Source CRDT based Data synchronization system
  2. 2. Applications start with a Database DB WRITES READS
  3. 3. Add a Write Through Cache DB Cache WRITES READS
  4. 4. Add Multiple Write Through Caches DB Cache WRITES READS Cache WRITES READS
  5. 5. Inevitable CONFLICTS DB Cache WRITES Cache WRITES
  6. 6. Datanet Agents MERGE conflicts DB Cache Cache Agent Agent Agent MERGE MERGE MERGE
  7. 7. Example
  8. 8. Start with X=5 X=5 X=5 X=5 Agent Agent Agent
  9. 9. Add 1 & 2 concurrently X=5 X=7 X=6 Agent Agent Agent ADD 1 ADD 2
  10. 10. Replication is done P2P X=5 X=7 X=8 Agent Agent Agent ADD 1 ADD 2
  11. 11. In Arbitrary Order X=5 X=8 X=8 Agent Agent Agent ADD 1 ADD 2
  12. 12. Replication to DB also X=8 X=8 X=8 Agent Agent Agent ADD 1 ADD 2
  13. 13. Architecture
  14. 14. Datanet supports many Agents DB Agent Agent Agent AgentAgent Agent Agent
  15. 15. Replication is P2P DB Agent Agent Agent AgentAgent Agent Agent
  16. 16. Central provides HighAvailability Agent Agent Central AgentAgent Agent Agent
  17. 17. Central is a HA Mesh Agent Agent AgentAgent Agent Agent Cluster Cluster Cluster
  18. 18. Datanet Replication Agent Agent AgentAgent Agent Agent Cluster Cluster Cluster
  19. 19. Robustness
  20. 20. Robustness to multiple failures Agent Agent AgentAgent Agent Agent Cluster Cluster Cluster
  21. 21. Writes while Agent temp Offline Agent Agent Agent Agent Agent Cluster Cluster Cluster Agent
  22. 22. Back Online Agent RESYNC Agent Agent Agent Agent Agent Cluster Cluster Cluster Agent
  23. 23. Seamless -> Steady State Agent Agent AgentAgent Agent Agent Cluster Cluster Cluster
  24. 24. Multiple Central Failure Agent Agent AgentAgent Agent Agent Cluster Cluster Cluster
  25. 25. Seamless -> Steady State Agent Agent AgentAgent Agent Agent Cluster
  26. 26. Elasicity: Add Central Agent Agent AgentAgent Agent Agent Cluster Cluster
  27. 27. New Use Cases
  28. 28. Global Distributed Data Cache for CDN Central PoP USA1 PoP USA2 PoP EUR1 PoP EUR2 PoP CHN1 PoP CHN2
  29. 29. HA Stateful AppServer Clusters Central Agent Frontend Request Sticky Routing 100% Local I/O AgentAgent CACHE Raft Quorum
  30. 30. Consistent Memcache Clusters Agent MEMCACHE Agent MEMCACHE App Server Cluster DC1 Agent MEMCACHE Agent MEMCACHE App Server Cluster DC2 Central
  31. 31. Big Picture
  32. 32. Ubiquitous Write Through Caching CacheCentral CacheCache Cache CacheCache
  33. 33. One Cache to Rule them ALL Cache Cache CentralCache Cache CDN PoP Multi DC Cache App Server Cluster Cache

×