Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Eventually-Consistent Data Structures

11,557 views

Published on

There are many reasons to use an eventually-consistent database -- like Riak, Voldemort, or Cassandra -- including increased availability, lower latency, and fault-tolerance. However, doing so requires a mental shift in how to structure client applications, and certain types of traditional data-structures, like sets, registers, and counters can't be resolved simply in the face of race-conditions. It is difficult to achieve "logical monotonicity" except for the most trivial data-types.
That is, until the advent of Conflict-Free Replicated Data Types (CRDTs). CRDTs are data-structures that tolerate eventual consistency. They replace traditional data-structure implementations and all have the property that, given any number of conflicting versions of the same datum, there is a single state on which they converge (monotonicity). This talk will discuss some of the most useful CRDTs and how to apply them to solve real-world data problems.

Published in: Technology, Business
  • Be the first to comment

Eventually-Consistent Data Structures

  1. 1. Eventually-Consistent Data Structures Sean Cribbs @seancribbs #CRDT Berlin Buzzwords 2012
  2. 2. I work for Basho We make
  3. 3. Riak is Eventually ConsistentSo are Voldemort and Cassandra
  4. 4. EventualConsistency Replicated Loose coordination 3 Forward progression
  5. 5. Eventual is Good ✔ Fault-tolerant ✔ Highly available ✔ Low-latency
  6. 6. Consistency? No clear winner! Throw one out? 3 Keep both?B
  7. 7. Consistency? No clear winner! Throw one out? 3 Keep both?B Cassandra
  8. 8. Consistency? No clear winner! Throw one out? 3 Keep both?B Cassandra Riak & Voldemort
  9. 9. Conflicts! A! B! Now what?
  10. 10. Semantic Resolution• Your app knows the domain - use business rules to resolve• Amazon Dynamo’s shopping cart
  11. 11. Semantic Resolution • Your app knows the domain - use business rules to resolve • Amazon Dynamo’s shopping cart“Ad hoc approaches have proven brittle and error-prone”
  12. 12. Conflict-Free Replicated Data Types
  13. 13. Conflict-Free Replicated Data Types useful abstractions
  14. 14. Conflict-Free Replicated Data Types multipleindependent copies useful abstractions
  15. 15. resolves automatically toward a single value Conflict-Free Replicated Data Types multipleindependent copies useful abstractions
  16. 16. CRDT Flavors• Convergent: State • Weak messaging requirements•Commutative: Operations •Reliable broadcast required •Causal ordering sufficient
  17. 17. Convergent CRDTs
  18. 18. Commutative CRDTs
  19. 19. RegistersA place to put your stuff
  20. 20. Registers• Last-Write Wins (LWW-Register) • e.g. Columns in Cassandra• Multi-Valued (MV-Register) • e.g. Objects (values) in Riak
  21. 21. Counters Keeping tabs
  22. 22. G-Counter
  23. 23. G-Counter// Starts empty[]
  24. 24. G-Counter// Starts empty[]// A increments twice, forwarding state[{a,1}] // == 1[{a,2}] // == 2
  25. 25. G-Counter// Starts empty[]// A increments twice, forwarding state[{a,1}] // == 1[{a,2}] // == 2 // B increments [{b,1}] // == 1
  26. 26. G-Counter// Starts empty[]// A increments twice, forwarding state[{a,1}] // == 1[{a,2}] // == 2 // B increments [{b,1}] // == 1// Merging[{a,2}, {b,1}] [{a,1}, {b,1}]
  27. 27. PN-Counter// A PN-Counter{ P = [{a,10},{b,2}], N = [{a,1},{c,5}]}// == (10+2)-(1+5) == 12-6 == 6
  28. 28. SetsMembers Only
  29. 29. G-Set
  30. 30. G-Set// Starts empty{}
  31. 31. G-Set// Starts empty{}// A adds a and b, forwarding state{a}{a,b}
  32. 32. G-Set// Starts empty{}// A adds a and b, forwarding state{a}{a,b} // B adds c {c}
  33. 33. G-Set// Starts empty{}// A adds a and b, forwarding state{a}{a,b} // B adds c {c}// Merging{a,b,c} {a,c}
  34. 34. 2P-Set
  35. 35. 2P-Set// Starts empty{A={},R={}}
  36. 36. 2P-Set// Starts empty{A={},R={}}// A adds a and b, forwarding state,// removes a{A={a}, R={}} // == {a}{A={a,b},R={}} // == {a,b}
  37. 37. 2P-Set// Starts empty{A={},R={}}// A adds a and b, forwarding state,// removes a{A={a}, R={}} // == {a}{A={a,b},R={}} // == {a,b}{A={a,b},R={a}} // == {b}
  38. 38. 2P-Set// Starts empty{A={},R={}}// A adds a and b, forwarding state,// removes a{A={a}, R={}} // == {a}{A={a,b},R={}} // == {a,b}{A={a,b},R={a}} // == {b} // B adds c
  39. 39. 2P-Set// Starts empty{A={},R={}}// A adds a and b, forwarding state,// removes a{A={a}, R={}} // == {a}{A={a,b},R={}} // == {a,b}{A={a,b},R={a}} // == {b} // B adds c {A={c},R={}} // == {c}
  40. 40. 2P-Set// Starts empty{A={},R={}}// A adds a and b, forwarding state,// removes a{A={a}, R={}} // == {a}{A={a,b},R={}} // == {a,b}{A={a,b},R={a}} // == {b} // B adds c {A={c},R={}} // == {c}// Merging
  41. 41. 2P-Set// Starts empty{A={},R={}}// A adds a and b, forwarding state,// removes a{A={a}, R={}} // == {a}{A={a,b},R={}} // == {a,b}{A={a,b},R={a}} // == {b} // B adds c {A={c},R={}} // == {c}// Merging{A={a,b,c},R={a}} {A={a,c}, R={}}
  42. 42. LWW-Element-Set
  43. 43. OR-Set
  44. 44. G = (V,E)Graphs E⊆V×V
  45. 45. G = (V,E)Graphs E⊆V×V
  46. 46. G = (V,E)Graphs E⊆V×V
  47. 47. Use-Cases• Social graph (OR-Set or a Graph)• Web page visits (G-Counter)• Shopping Cart (Modified OR-Set)• “Like” button (U-Set)
  48. 48. Challenges: GC• CRDTs are inefficient• Synchronization may be required
  49. 49. Challenges: Responsibility• Client • Erlang: mochi/statebox • Clojure: reiddraper/knockbox • Ruby: aphyr/meangirls• Server • Very few options
  50. 50. Thanks

×