The document discusses conflict-free replicated data types (CRDTs) as a way to achieve strong eventual consistency in distributed systems. It explains that CRDTs allow for concurrent updates by using operation-based or state-based approaches and defining merge functions that ensure convergence to a consistent state. Examples of CRDT data structures like grow-only sets, counters and maps are provided to illustrate how conflicts are resolved during replication across nodes. Industry usage of CRDTs in databases from Redis, Riak, Facebook and Microsoft is also mentioned.

1. Tale of a
growing
startup
Praveen Singh Bora
Rajesh Kumar

2. Once upon a
time…

3. There was
a koala

4. And I was in vicinity too, So I did the
only thing that came immediately to
my mind

5. I shot it…

6. With the
camera of
course

7. I of course uploaded the
pictures and they were a huge
hit
I created my own
website

8. Koality

9. Very first
problem
I wanted to
store likes of a
particular
picture

10. It was not hard and I
stored the likes counter
in RDBMS 42

11. Soon the
usage
soared all
over the
world

12. o Master Slave
architecture
o Master Master
architecture aka Multi
Master
Replicate
d the
instances
all over
the world.

13. Still RDBMS is not able to keep up

14. RDBMS provide active
active architecture, but
with 2PC

15. o Blocking architecture
o 2PC is notoriously
slow.
o 2PC can get stuck in
a transaction when
coordinator dies.
2PC
Limitatio
ns

16. Who wants to be seen doing
RDBMS these days?

17. Considering NoSQLs

18. Can I safely say scaling is not a problem with
NoSQLs?

19. What about
data
consistency?

20. Strong consistency
Eventual consistency

21. o Events are processed as
soon as they happen
o All or nothing operation
o ACID Compliant
o Also known as
linearizability
o No concurrent updates
o E.g. XA transactions
Strong
Consisten
cy

22. o Events are not eventually
processed
o Exchanges versions of
data
o Reconciling final state
o System may be in
inconsistent state but
eventually will be consistent
o Allows concurrent updates
o E.g. Peer to peer systems,
No Sql databases etc
Eventual
Consisten
cy

23. A lot to
choose
from?

24. But which of them
are matured and
asynchronous
master-less
replication?

25. o Last Writer Wins
o CRDTs
o Operational
Transformation
no sql
Conflict
Resolutio
n
Strategies

26. Add +1
TS: T1
Add +9
TS: T2
200C
200C
201C
209C
Last Writer
Wins
209C
Value 201
Discarded

27. We have
Data
Consistency
issues

28. What we
learnt from
our
endeavor
so far

29. If you tried
your best and
failed
Lesson is,
NEVER TRY

30. CRDTs provide a way for you
to merge concurrent
modifications, always, in
any order.
e.g. Set Union, Max
operation on number
Conflict
Free
Replicated
Data
Types

31. o Special case of
Eventual consistency
o Valid only for certain
data types known as
Conflict Free Replicated
Data types aka CRDTs
Strong
Eventual
Consisten
cy

32. o Convergent replicated data types
o Sends full local state in operation
hence called state based CRDTs
o Commutative: A ✧ B = B ✧ A
o Associative: A ✧ (B ✧ C) = (A ✧ B)
✧ C
o Idempotent: A ✧ A = A
CvRDTs

33. o Commutative replicated data
types
o Only transmit the update
part hence called operation
based CRDTs
o Commutative: A ✧ B = B ✧
A
CmRDTs

34. Join
Semilattice
o A graph that converges in on
a single ascendant
o This single ascendant is
known as the least upper
bound
o Two objects can be either
equal, have hierarchy or are
pairs

35. { 1, 0,
0 }
{ 0, 1,
0 }
{ 0, 0,
1 }
{ 0, 0,
0 }
{ 1, 1,
0 }
{ 0, 1,
1 }
{ 1, 1,
1 }
Merging of
states

36. Grow
Only
Set

37. Add = {AD1, AD2}
Rem = {AD2}A B
C
Add = {AD1, AD2}
Rem = {AD2}
Add = {AD1, AD2}
Rem = {AD2}
Rem
AD1
Add
AD4
Add
AD3
Concurrent
Updates
On
All Nodes
Initial
State

38. Add = {AD1, AD2}
Rem = {AD2, AD1}A B
C
Add = {AD1, AD2,
AD3}
Rem = {AD2}
Add = {AD1, AD2,
AD4}
Rem = {AD2}
Immediate
Nodes
Updated

39. Add = {AD1, AD2}
Rem = {AD2, AD1}A B
C
Add = {AD1, AD2,
AD3}
Rem = {AD2}
Add = {AD1, AD2,
AD4}
Rem = {AD2}
Nodes
Syncing
Add = {AD1, AD2}
Rem = {AD2, AD1}
Add = {AD1, AD2,
AD3}
Rem = {AD2}
Add = {AD1, AD2,
AD4}
Rem = {AD2}
Add = {AD1, AD2,
AD4}
Rem = {AD2}
Add = {AD1, AD2}
Rem = {AD2, AD1}
Add = {AD1, AD2,
AD3}
Rem = {AD2}

40. Add = {AD1, AD2,
AD3, AD4}
Rem = {AD1, AD2}
A B
C
Add = {AD1, AD2,
AD3, AD4}
Rem = {AD1, AD2}
Add = {AD1, AD2,
AD3, AD4}
Rem = {AD1, AD2}
Nodes
Synced

41. Revisit
the
counter

42. o Grow counter is a value
that can be
incremented by any
positive integer.
o All nodes keep version
vector values.
o The merge function
while reconciliation
takes the max value of
version.
Grow
Only
Counter

43. A = {0,
0}
B = {0,
0}
C = {0,
0}
A = {0,
0}
B = {0,
0}
C = {0,
0}
A = {0,
0}
B = {0,
0}
C = {0,
0}
A B
C
Initial
State
incr 2 incr 3
incr 5
Concurren
t
Updates
Received

44. Immediate
Nodes
Updated
A = {2,
1}
B = {0,
0}
C = {0,
0}
A = {0,
0}
B = {3,
1}
C = {0,
0}
A = {0,
0}
B = {0,
0}
C = {5,
1}
A B
C
A = {2,
1}
B = {0,
0}
C = {0,
0}
A = {0,
0}
B = {3,
1}
C = {0,
0}
A = {0,
0}
B = {0,
0}
C = {5,
1}
A = {0,
0}
B = {0,
0}
C = {5,
1}
A = {2,
1}
B = {0,
0}
C = {0,
0}
A = {0,
0}
B = {3,
1}
C = {0,
0}
Nodes
Syncing

45. A = {2,
1}
B = {3,
1}
C = {5,
1}
A = {2,
1}
B = {3,
1}
C = {5,
1}
A = {2,
1}
B = {3,
1}
C = {5,
1}
A B
C
Nodes
Synced

46. CRDT
Design
Rules
o New values must always
be greater than previous
values
o Conflicting updates
must always produce
sibling values
o Resolution must always
result in values greater
than participants

47. o Last writer wins Set
o OR Sets
o Observed Remove Map
o Grow Only Map
o Last Writer Wins Map
o Add only Monotonic
DAG
o Replicated Growable
Array
Other
Data
Structures

48. o Redis uses CRDTs for
distributed databases.
o SoundCloud Roshi, a
LWW-element-set on top
of Redis.
o Riak is a distributed key-
value data store based on
CRDTs.
o TomTom employs CRDTs
to synchronize navigation
data.
Industry
Usage

49. o Facebook implements
CRDTs in their Apollo
database.
o Teletype for Atom
employs CRDTs.
o Microsoft's Cosmos DB
uses CRDT's to enable a
multi-master write
mode.
Industry
Usage
Continued