Eventual Consitency with CRDTS
•Download as PPTX, PDF•
2 likes•644 views
The document discusses the use of CRDTs (Convergent Replicated Data Types) to achieve eventual consistency in distributed systems without consensus. It describes the CAP theorem and challenges with achieving consistency in a distributed manner. CRDTs are introduced as a way to build datatypes that can automatically resolve conflicts as they propagate through replicas. Examples of commonly used CRDTs include registers, counters, sets and graphs. The document outlines some real-world implementations of CRDTs and notes their limitations.
Recommended
More Related Content
Viewers also liked
Viewers also liked (18)
Similar to Eventual Consitency with CRDTS
Similar to Eventual Consitency with CRDTS (16)
More from Samir Bessalah
More from Samir Bessalah (9)
Recently uploaded
Recently uploaded (20)
Eventual Consitency with CRDTS
- 1. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Eventual Consistency without Consensus with CRDTs Sam BESSALAH - @samklr
- 2. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT “ This talk is filled with words and terms, that might make you sound too nerdy or pedantic at dinners with non developers friends. ” Use with caution.
- 3. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT “ Distributed Programming, generally a bad idea, best avoided. ” -Peter Bourgon
- 6. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT - Availability - Fault tolerance - Throughtput - Architecture - Economics Why we use distributed systems?
- 8. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CAP Theorem S. Gilbert and N. A. Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News, 33(2):51–59, 2002.
- 9. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT http://bravenewgeek.com/from-mainframe-to-microservice-an-introduction-to-distributed-systems/
- 10. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Data Consistency? Consensus Systems , Locking Services and “barbaric“ algorithms
- 11. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Data Consistency? - Distributed Consensus : Costly and nearly impossible. Multi Phase Commit, Stae Replication - Two Phase Commit : Blocking, dependent on coordinator, deadlocks inducing - Three Phase commit : abort on timeouts, non blocking but easily fails on network partitions
- 12. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Data Consistency?
- 13. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Data Consistency?
- 14. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Data Consistency?
- 15. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Data Consistency? RAFT
- 16. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Data Consistency? Locking Services Chubby (Google) Zab (Yahoo)
- 17. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT http://bravenewgeek.com/from-mainframe-to-microservice-an-introduction-to-distributed-systems/
- 18. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT @aphyr
- 19. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT “ Consistency is a property of your data, not of your nodes” @aphyr
- 20. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT AP Systems Eventual Consistency
- 21. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT AP Systems Eventual Consistency - High Availability - Low Latency - Fault tolerant
- 22. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT AP Systems Eventual Consistency - High Availability - Low Latency - Fault tolerant
- 23. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Dynamo Systems Riak Voldemort Cassandra
- 25. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Conflicts resolution - Semantic Resolution - Vector Clocks - Last write wins (LWW)
- 26. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Vector ClocksLWW
- 27. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Convergent Replicated Data Types
- 29. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CRDTs - Take the consistency problem to the level of Data Structures - Their state resolves automatically (eventually) to a single coherent value - Maintain multiple copies of your data
- 30. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CRDTs : Monotonicity - Every new operation adds information - Data is never immediately destroyed - Most things are trasparent to the application
- 31. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CRDTs : 2 types - State based or convergent ( CvRDTs) - Commutative or Operations based (CmRDTs) *
- 32. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CvRDTs - All replicas connected - At least once semantics usually - State Changes advance upwards according to partial order
- 33. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CmRDTs - All replicas connected - Need Reliable broadcast with ordering guarantees - Best suitable for commutative updates
- 34. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CRDTs : Fancy words CRDTs are idempotent, commutative monoids !!!
- 35. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CRDTs : Fancy words CRDTs are idempotent, commutative monoid !!! aka Joint Semi Lattice
- 36. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT - Idempotence : A * A = A - Commutative : A*B = B* A - Associative : A*(B*C) = (A*B)*C Joint SemiLattice
- 37. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CvRDTs
- 38. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Some common CRDTs - Registers (LWW, Multi Valued Register) - G-Counter - PN Counter - G-Set, 2P-Set and OR-Set - Graph - Maps
- 39. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT G- Counter : grow only counter
- 40. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT PN-Counter - Positive and Negative Counters - Uses two G-counter - One for increments(P) and another for decrement(N) - Result is the difference
- 41. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT G-Set Grow only set, that only allows to add an element
- 42. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT 2P-Set - Two Phase Set - Built with 2 G-set for add and removal - Can’t add an already removed element : Tombstone set - Has a tombstone that maintains deleted elements
- 43. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT LWW-Element-set - Add Timestamp to “add” and “remove” states wit h a timestamp. - Greatest timestamps wins - Close to cassandra model
- 44. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT CRDTs in the Wild
- 46. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT www.jepsen.io - Riak - Cassandra - Kafka - etc.
- 47. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Roshi https://github.com/soundcloud/roshi
- 48. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Eventuate
- 49. @YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT Limitations - Garbage Collection - Not always easy to have Semi Lattice for all use case - Inducing some strange behaviours - Might require adding a stronger consistency models: Raft, Paxos, etc - Use when availability is really important - Don’t use them for your billing application - Definitely not a panacea
Editor's Notes
- availability : every req gets a response consistency all nodes see the same data at the same time Partition tolerance system keeps going despite failure or message loss
- availability : every req gets a response consistency all nodes see the same data at the same time Partition tolerance system keeps going despite failure or message loss
- Linearizability: single-operation, single-object, real-time order Serializability: multi-operation, multi-object, arbitrary total order
- availability : every req gets a response consistency all nodes see the same data at the same time Partition tolerance system keeps going despite failure or message loss
- Semantic Resolution : Let business decide Vector Clock : A vector clock (vclock) is a system for tracking the causality of concurrent updates to a piece of data.
- P and Negative Counter
- Hard to make them converge Can build other data types on top of them Only two operations : Add an element and Remove
- 2 Phase
- 2 Phase
- Casandra Columns Cassandra collections Riak Data type : PN-counter