This talk was due to be presented at JAX London 2013, but the speaker was unfortunately unable to attend. Distributed data stores give us increased availability, linear scalability, predictable latency and improved fault tolerance. The flip-side is having to deal with inconsistencies: most distributed databases will ask your application layer how to resolve such inconsistencies. Conflict-free Replicated Data Types (CRDTs) are a way for a distributed database, such as Riak, to resolve those inconsistencies logically and automatically. Unlike traditional data structures, there is always a single state on which they converge. In this talk, I’ll look at the development of CRDTs from an academic project to implementation in Riak.

1. Conflict Free Replicated
Data-types in Eventually
Consistent Systems
Joel Jacobson, Basho Technologies
JAX London
2013
Friday, 8 November 13

2. Retweet to win a copy!
@joeljacobson
Friday, 8 November 13

3. Riak
Distributed, Masterless, Key/Value Database
+ Extras
Friday, 8 November 13

4. Key
Value
Key
Value
Key
Value
Key
Value
Key
Value
Key
Value
Buckets, Keys and Values
Simple operations;
GET, PUT, DELETE
Friday, 8 November 13

5. Distributed & Scalable
Friday, 8 November 13

6. Distributed & Scalable
Friday, 8 November 13

7. Fault Tolerance
Any node can serve R/W requests
Data is replicated
Fallback Node
Hinted Handoff
Friday, 8 November 13

8. Dynamo Systems
Distributed
High Availability
Masterless
Eventually Consistent
Friday, 8 November 13

9. Client
A
1
2
3
Friday, 8 November 13

10. Client
A
1
2
3
Friday, 8 November 13

11. N
Client
1
2
N
Client
Friday, 8 November 13
3

12. Conflicts
Friday, 8 November 13

13. Conflict Resolution
Last-Write Wins (LWW)
Vector Clocks (Siblings)
Friday, 8 November 13

14. Sibling Resolution
Friday, 8 November 13

15. Conflict-Free Replicated
Data Types
Friday, 8 November 13

16. http://hal.upmc.fr/docs/00/55/55/88/PDF/
techreport.pdf
Friday, 8 November 13

17. Primitives
Counters
Sets
Maps
Registers
Booleans
Friday, 8 November 13

18. Counters
Friday, 8 November 13

19. Counters
{
'type':
'g-­‐
counter',
'e':
{
'a':
1,
'b':
5,
'c':
2
}
}
Friday, 8 November 13

20. Counters
{
'type':
'g-­‐
counter',
'e':
{
'a':
1,
'b':
5,
'c':
2
}
}
Friday, 8 November 13
{
'type':
'pn-­‐counter',
'p':
{
'a':
10,
'b':
2
},
'n':
{
'c':
5,
'a':
1
}
}

21. Counters
{
'type':
'g-­‐
counter',
'e':
{
'a':
1,
'b':
5,
'c':
2
}
}
{
'type':
'pn-­‐counter',
'p':
{
'a':
10,
'b':
2
},
'n':
{
'c':
5,
'a':
1
}
}
P=12, N=6, so the value is 6
Friday, 8 November 13

22. The Problem of Absence
Friday, 8 November 13

23. Sets
Friday, 8 November 13

24. Sets
{
'type':
'g-­‐set',
'e':
['a',
'b',
'c']
}
Friday, 8 November 13

25. Sets
{
'type':
'g-­‐set',
'e':
['a',
'b',
'c']
}
'type':
'or-­‐set'
{
[
{a,
3},
{b,
2},
{c,
2}],
%%
Version
Vector
[{
'bob',
[{a,
1}]},
%%
Bob
was
added
by
'a'
at
logical
time
1
(a's
first
event)
{'joel',
[{a,
3}]},
%%
Joel
was
added
by
'a'
at
logical
time
3
and
by
'c'
at
logical
time
1
{'sean',
[{a,
2},
{c,
1}]},
{'matt',
[{b,
1}]},
{'stu',
[{c,
2},
{b,
2}]}]
]}
Friday, 8 November 13

26. Sets
'type':
'or-­‐set'
{
[
{a,
3},
{b,
2},
{c,
2}],
%%
Version
Vector
[{
'bob',
[{a,
1}]},
%%
Bob
was
added
by
'a'
at
logical
time
1
(a's
first
event)
{'joel',
[{a,
3}]},
%%
Joel
was
added
by
'a'
at
logical
time
3
and
by
'c'
at
logical
time
1
{'sean',
[{a,
2},
{c,
1}]},
{'matt',
[{b,
1}]},
{'stu',
[{c,
2},
{b,
2}]}]
]}
Friday, 8 November 13

27. Sets
'type':
'or-­‐set'
{
[
{a,
3},
{b,
2},
{c,
2}],
%%
Version
Vector
[{
'bob',
[{a,
1}]},
%%
Bob
was
added
by
'a'
at
logical
time
1
(a's
first
event)
{'joel',
[{a,
3}]},
%%
Joel
was
added
by
'a'
at
logical
time
3
and
by
'c'
at
logical
time
1
{'sean',
[{a,
2},
{c,
1}]},
{'matt',
[{b,
1}]},
VV,
element-­‐>dots
{'stu',
[{c,
2},
{b,
2}]}]
]}
Friday, 8 November 13

28. Maps
{
'type':
'map',
gold=100,
stone
=
10,
weapons=[sword,
knife,
shotgun],
armour=[helmet,
shield]
}
Friday, 8 November 13

29. Maps
{
'type':
'map',
gold=100,
stone
=
10,
weapons=[sword,
knife,
shotgun],
armour=[helmet,
shield]
}
Hey
Riak
with
Map
at
Key
K
-­‐>
[{increment
gold
by
10,
decrement
stone
by
4,
add
uzi
to
weapons
and
remove
helmet
from
armour}]
Friday, 8 November 13

30. Use-Cases
Advert clicks (G-Counter)
Shopping cart (Modified OR-Set)
Logged in users (P-N Counter)
Maps can be composed of complex data
Friday, 8 November 13

31. Common Questions
What do they cost me?
How big are they?
What can’t they do?
Friday, 8 November 13

32. Useful links
http://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf
http://arxiv.org/pdf/1210.3368.pdf
https://gist.github.com/russelldb/f92f44bdfb619e089a4d
http://gsd.di.uminho.pt/members/cbm/ps/scadt3.pdf
http://arxiv.org/abs/1011.5808
Friday, 8 November 13

33. Thank you
joel@basho.com
Friday, 8 November 13