Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel Jacobson (Basho Technologies)
Upcoming SlideShare
Loading in...5
×
 

Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel Jacobson (Basho Technologies)

on

  • 2,696 views

This talk was due to be presented at JAX London 2013, but the speaker was unfortunately unable to attend. ...

This talk was due to be presented at JAX London 2013, but the speaker was unfortunately unable to attend.

Distributed data stores give us increased availability, linear scalability, predictable latency and improved fault tolerance. The flip-side is having to deal with inconsistencies: most distributed databases will ask your application layer how to resolve such inconsistencies. Conflict-free Replicated Data Types (CRDTs) are a way for a distributed database, such as Riak, to resolve those inconsistencies logically and automatically. Unlike traditional data structures, there is always a single state on which they converge.

In this talk, I’ll look at the development of CRDTs from an academic project to implementation in Riak.

Statistics

Views

Total Views
2,696
Views on SlideShare
2,692
Embed Views
4

Actions

Likes
0
Downloads
7
Comments
0

1 Embed 4

https://twitter.com 4

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel Jacobson (Basho Technologies) Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel Jacobson (Basho Technologies) Presentation Transcript

  • Conflict Free Replicated Data-types in Eventually Consistent Systems Joel Jacobson, Basho Technologies JAX London 2013 Friday, 8 November 13
  • Retweet to win a copy! @joeljacobson Friday, 8 November 13
  • Riak Distributed, Masterless, Key/Value Database + Extras Friday, 8 November 13
  • Key Value Key Value Key Value Key Value Key Value Key Value Buckets, Keys and Values Simple operations; GET, PUT, DELETE Friday, 8 November 13
  • Distributed & Scalable Friday, 8 November 13
  • Distributed & Scalable Friday, 8 November 13
  • Fault Tolerance Any node can serve R/W requests Data is replicated Fallback Node Hinted Handoff Friday, 8 November 13
  • Dynamo Systems Distributed High Availability Masterless Eventually Consistent Friday, 8 November 13
  • Client A 1 2 3 Friday, 8 November 13
  • Client A 1 2 3 Friday, 8 November 13
  • N Client 1 2 N Client Friday, 8 November 13 3
  • Conflicts Friday, 8 November 13
  • Conflict Resolution Last-Write Wins (LWW) Vector Clocks (Siblings) Friday, 8 November 13
  • Sibling Resolution Friday, 8 November 13
  • Conflict-Free Replicated Data Types Friday, 8 November 13
  • http://hal.upmc.fr/docs/00/55/55/88/PDF/ techreport.pdf Friday, 8 November 13
  • Primitives Counters Sets Maps Registers Booleans Friday, 8 November 13
  • Counters Friday, 8 November 13
  • Counters {    'type':  'g-­‐ counter',    'e':  {        'a':  1,        'b':  5,        'c':  2    } } Friday, 8 November 13
  • Counters {    'type':  'g-­‐ counter',    'e':  {        'a':  1,        'b':  5,        'c':  2    } } Friday, 8 November 13 {    'type':  'pn-­‐counter',    'p':  {        'a':  10,        'b':  2    },  'n':  {        'c':  5,        'a':  1    } }
  • Counters {    'type':  'g-­‐ counter',    'e':  {        'a':  1,        'b':  5,        'c':  2    } } {    'type':  'pn-­‐counter',    'p':  {        'a':  10,        'b':  2    },  'n':  {        'c':  5,        'a':  1    } } P=12, N=6, so the value is 6 Friday, 8 November 13
  • The Problem of Absence Friday, 8 November 13
  • Sets Friday, 8 November 13
  • Sets {    'type':  'g-­‐set',    'e':  ['a',  'b',  'c'] } Friday, 8 November 13
  • Sets {    'type':  'g-­‐set',    'e':  ['a',  'b',  'c'] } 'type':  'or-­‐set' {   [  {a,  3},  {b,  2},  {c,  2}],                %%  Version  Vector    [{  'bob',  [{a,  1}]},                            %%  Bob  was  added  by   'a'  at  logical  time  1  (a's  first  event)    {'joel',    [{a,  3}]},                            %%  Joel  was  added  by   'a'  at  logical  time  3  and  by  'c'  at  logical  time  1    {'sean',  [{a,  2},  {c,  1}]},                    {'matt',  [{b,  1}]},    {'stu',  [{c,  2},  {b,  2}]}] ]} Friday, 8 November 13
  • Sets 'type':  'or-­‐set' {   [  {a,  3},  {b,  2},  {c,  2}],                %%  Version  Vector    [{  'bob',  [{a,  1}]},                            %%  Bob  was  added  by   'a'  at  logical  time  1  (a's  first  event)    {'joel',    [{a,  3}]},                            %%  Joel  was  added  by   'a'  at  logical  time  3  and  by  'c'  at  logical  time  1    {'sean',  [{a,  2},  {c,  1}]},                    {'matt',  [{b,  1}]},    {'stu',  [{c,  2},  {b,  2}]}] ]} Friday, 8 November 13
  • Sets 'type':  'or-­‐set' {   [  {a,  3},  {b,  2},  {c,  2}],                %%  Version  Vector    [{  'bob',  [{a,  1}]},                            %%  Bob  was  added  by   'a'  at  logical  time  1  (a's  first  event)    {'joel',    [{a,  3}]},                            %%  Joel  was  added  by   'a'  at  logical  time  3  and  by  'c'  at  logical  time  1    {'sean',  [{a,  2},  {c,  1}]},                    {'matt',  [{b,  1}]}, VV,  element-­‐>dots    {'stu',  [{c,  2},  {b,  2}]}] ]} Friday, 8 November 13
  • Maps {    'type':  'map',              gold=100,  stone  =  10,                weapons=[sword,  knife,   shotgun],              armour=[helmet,  shield] } Friday, 8 November 13
  • Maps {    'type':  'map',              gold=100,  stone  =  10,                weapons=[sword,  knife,   shotgun],              armour=[helmet,  shield] } Hey  Riak  with  Map  at  Key  K  -­‐>   [{increment  gold  by  10,  decrement   stone  by  4,  add  uzi  to  weapons  and   remove  helmet  from  armour}] Friday, 8 November 13
  • Use-Cases Advert clicks (G-Counter) Shopping cart (Modified OR-Set) Logged in users (P-N Counter) Maps can be composed of complex data Friday, 8 November 13
  • Common Questions What do they cost me? How big are they? What can’t they do? Friday, 8 November 13
  • Useful links http://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf http://arxiv.org/pdf/1210.3368.pdf https://gist.github.com/russelldb/f92f44bdfb619e089a4d http://gsd.di.uminho.pt/members/cbm/ps/scadt3.pdf http://arxiv.org/abs/1011.5808 Friday, 8 November 13
  • Thank you joel@basho.com Friday, 8 November 13