0
Conflict Free Replicated
Data-types in Eventually
Consistent Systems
Joel Jacobson, Basho Technologies
JAX London
2013

Fri...
Retweet to win a copy!
@joeljacobson

Friday, 8 November 13
Riak
Distributed, Masterless, Key/Value Database
+ Extras
Friday, 8 November 13
Key

Value

Key

Value

Key

Value

Key

Value

Key

Value

Key

Value

Buckets, Keys and Values
Simple operations;
GET, P...
Distributed & Scalable

Friday, 8 November 13
Distributed & Scalable

Friday, 8 November 13
Fault Tolerance
Any node can serve R/W requests
Data is replicated
Fallback Node
Hinted Handoff

Friday, 8 November 13
Dynamo Systems
Distributed
High Availability
Masterless
Eventually Consistent

Friday, 8 November 13
Client

A

1
2
3

Friday, 8 November 13
Client

A

1
2
3

Friday, 8 November 13
N

Client

1
2

N
Client

Friday, 8 November 13

3
Conflicts

Friday, 8 November 13
Conflict Resolution

Last-Write Wins (LWW)
Vector Clocks (Siblings)

Friday, 8 November 13
Sibling Resolution

Friday, 8 November 13
Conflict-Free Replicated
Data Types

Friday, 8 November 13
http://hal.upmc.fr/docs/00/55/55/88/PDF/
techreport.pdf
Friday, 8 November 13
Primitives
Counters
Sets
Maps
Registers
Booleans

Friday, 8 November 13
Counters

Friday, 8 November 13
Counters
{
	
  	
  'type':	
  'g-­‐
counter',
	
  	
  'e':	
  {
	
  	
  	
  	
  'a':	
  1,
	
  	
  	
  	
  'b':	
  5,
	
  ...
Counters
{
	
  	
  'type':	
  'g-­‐
counter',
	
  	
  'e':	
  {
	
  	
  	
  	
  'a':	
  1,
	
  	
  	
  	
  'b':	
  5,
	
  ...
Counters
{
	
  	
  'type':	
  'g-­‐
counter',
	
  	
  'e':	
  {
	
  	
  	
  	
  'a':	
  1,
	
  	
  	
  	
  'b':	
  5,
	
  ...
The Problem of Absence

Friday, 8 November 13
Sets

Friday, 8 November 13
Sets
{
	
  	
  'type':	
  'g-­‐set',
	
  	
  'e':	
  ['a',	
  'b',	
  'c']
}

Friday, 8 November 13
Sets
{
	
  	
  'type':	
  'g-­‐set',
	
  	
  'e':	
  ['a',	
  'b',	
  'c']
}
'type':	
  'or-­‐set'
{	
  
[	
  {a,	
  3},	
...
Sets
'type':	
  'or-­‐set'
{	
  
[	
  {a,	
  3},	
  {b,	
  2},	
  {c,	
  2}],	
  	
  	
  	
  	
  	
  	
  	
  %%	
  Version...
Sets
'type':	
  'or-­‐set'
{	
  
[	
  {a,	
  3},	
  {b,	
  2},	
  {c,	
  2}],	
  	
  	
  	
  	
  	
  	
  	
  %%	
  Version...
Maps
{
	
  	
  'type':	
  'map',
	
  	
  	
  	
  	
  	
  	
  gold=100,	
  stone	
  =	
  10,	
  
	
  	
  	
  	
  	
  	
  	
...
Maps
{
	
  	
  'type':	
  'map',
	
  	
  	
  	
  	
  	
  	
  gold=100,	
  stone	
  =	
  10,	
  
	
  	
  	
  	
  	
  	
  	
...
Use-Cases

Advert clicks (G-Counter)
Shopping cart (Modified OR-Set)
Logged in users (P-N Counter)
Maps can be composed of ...
Common Questions

What do they cost me?
How big are they?
What can’t they do?

Friday, 8 November 13
Useful links
http://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf
http://arxiv.org/pdf/1210.3368.pdf
https://gist.github...
Thank you
joel@basho.com

Friday, 8 November 13
Upcoming SlideShare
Loading in...5
×

Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel Jacobson (Basho Technologies)

2,969

Published on

This talk was due to be presented at JAX London 2013, but the speaker was unfortunately unable to attend.

Distributed data stores give us increased availability, linear scalability, predictable latency and improved fault tolerance. The flip-side is having to deal with inconsistencies: most distributed databases will ask your application layer how to resolve such inconsistencies. Conflict-free Replicated Data Types (CRDTs) are a way for a distributed database, such as Riak, to resolve those inconsistencies logically and automatically. Unlike traditional data structures, there is always a single state on which they converge.

In this talk, I’ll look at the development of CRDTs from an academic project to implementation in Riak.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,969
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel Jacobson (Basho Technologies)"

  1. 1. Conflict Free Replicated Data-types in Eventually Consistent Systems Joel Jacobson, Basho Technologies JAX London 2013 Friday, 8 November 13
  2. 2. Retweet to win a copy! @joeljacobson Friday, 8 November 13
  3. 3. Riak Distributed, Masterless, Key/Value Database + Extras Friday, 8 November 13
  4. 4. Key Value Key Value Key Value Key Value Key Value Key Value Buckets, Keys and Values Simple operations; GET, PUT, DELETE Friday, 8 November 13
  5. 5. Distributed & Scalable Friday, 8 November 13
  6. 6. Distributed & Scalable Friday, 8 November 13
  7. 7. Fault Tolerance Any node can serve R/W requests Data is replicated Fallback Node Hinted Handoff Friday, 8 November 13
  8. 8. Dynamo Systems Distributed High Availability Masterless Eventually Consistent Friday, 8 November 13
  9. 9. Client A 1 2 3 Friday, 8 November 13
  10. 10. Client A 1 2 3 Friday, 8 November 13
  11. 11. N Client 1 2 N Client Friday, 8 November 13 3
  12. 12. Conflicts Friday, 8 November 13
  13. 13. Conflict Resolution Last-Write Wins (LWW) Vector Clocks (Siblings) Friday, 8 November 13
  14. 14. Sibling Resolution Friday, 8 November 13
  15. 15. Conflict-Free Replicated Data Types Friday, 8 November 13
  16. 16. http://hal.upmc.fr/docs/00/55/55/88/PDF/ techreport.pdf Friday, 8 November 13
  17. 17. Primitives Counters Sets Maps Registers Booleans Friday, 8 November 13
  18. 18. Counters Friday, 8 November 13
  19. 19. Counters {    'type':  'g-­‐ counter',    'e':  {        'a':  1,        'b':  5,        'c':  2    } } Friday, 8 November 13
  20. 20. Counters {    'type':  'g-­‐ counter',    'e':  {        'a':  1,        'b':  5,        'c':  2    } } Friday, 8 November 13 {    'type':  'pn-­‐counter',    'p':  {        'a':  10,        'b':  2    },  'n':  {        'c':  5,        'a':  1    } }
  21. 21. Counters {    'type':  'g-­‐ counter',    'e':  {        'a':  1,        'b':  5,        'c':  2    } } {    'type':  'pn-­‐counter',    'p':  {        'a':  10,        'b':  2    },  'n':  {        'c':  5,        'a':  1    } } P=12, N=6, so the value is 6 Friday, 8 November 13
  22. 22. The Problem of Absence Friday, 8 November 13
  23. 23. Sets Friday, 8 November 13
  24. 24. Sets {    'type':  'g-­‐set',    'e':  ['a',  'b',  'c'] } Friday, 8 November 13
  25. 25. Sets {    'type':  'g-­‐set',    'e':  ['a',  'b',  'c'] } 'type':  'or-­‐set' {   [  {a,  3},  {b,  2},  {c,  2}],                %%  Version  Vector    [{  'bob',  [{a,  1}]},                            %%  Bob  was  added  by   'a'  at  logical  time  1  (a's  first  event)    {'joel',    [{a,  3}]},                            %%  Joel  was  added  by   'a'  at  logical  time  3  and  by  'c'  at  logical  time  1    {'sean',  [{a,  2},  {c,  1}]},                    {'matt',  [{b,  1}]},    {'stu',  [{c,  2},  {b,  2}]}] ]} Friday, 8 November 13
  26. 26. Sets 'type':  'or-­‐set' {   [  {a,  3},  {b,  2},  {c,  2}],                %%  Version  Vector    [{  'bob',  [{a,  1}]},                            %%  Bob  was  added  by   'a'  at  logical  time  1  (a's  first  event)    {'joel',    [{a,  3}]},                            %%  Joel  was  added  by   'a'  at  logical  time  3  and  by  'c'  at  logical  time  1    {'sean',  [{a,  2},  {c,  1}]},                    {'matt',  [{b,  1}]},    {'stu',  [{c,  2},  {b,  2}]}] ]} Friday, 8 November 13
  27. 27. Sets 'type':  'or-­‐set' {   [  {a,  3},  {b,  2},  {c,  2}],                %%  Version  Vector    [{  'bob',  [{a,  1}]},                            %%  Bob  was  added  by   'a'  at  logical  time  1  (a's  first  event)    {'joel',    [{a,  3}]},                            %%  Joel  was  added  by   'a'  at  logical  time  3  and  by  'c'  at  logical  time  1    {'sean',  [{a,  2},  {c,  1}]},                    {'matt',  [{b,  1}]}, VV,  element-­‐>dots    {'stu',  [{c,  2},  {b,  2}]}] ]} Friday, 8 November 13
  28. 28. Maps {    'type':  'map',              gold=100,  stone  =  10,                weapons=[sword,  knife,   shotgun],              armour=[helmet,  shield] } Friday, 8 November 13
  29. 29. Maps {    'type':  'map',              gold=100,  stone  =  10,                weapons=[sword,  knife,   shotgun],              armour=[helmet,  shield] } Hey  Riak  with  Map  at  Key  K  -­‐>   [{increment  gold  by  10,  decrement   stone  by  4,  add  uzi  to  weapons  and   remove  helmet  from  armour}] Friday, 8 November 13
  30. 30. Use-Cases Advert clicks (G-Counter) Shopping cart (Modified OR-Set) Logged in users (P-N Counter) Maps can be composed of complex data Friday, 8 November 13
  31. 31. Common Questions What do they cost me? How big are they? What can’t they do? Friday, 8 November 13
  32. 32. Useful links http://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf http://arxiv.org/pdf/1210.3368.pdf https://gist.github.com/russelldb/f92f44bdfb619e089a4d http://gsd.di.uminho.pt/members/cbm/ps/scadt3.pdf http://arxiv.org/abs/1011.5808 Friday, 8 November 13
  33. 33. Thank you joel@basho.com Friday, 8 November 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×