Eventual Consitency with CRDTS

@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #CRDT
Eventual Consistency without Consensus with
CRDTs
Sam BESSALAH - @samklr

“ This talk is filled with words and terms, that might
make you sound too nerdy or pedantic at dinners
with non developers friends. ”
Use with caution.

“ Distributed Programming, generally a bad idea,
best avoided. ”
-Peter Bourgon

- Availability
- Fault tolerance
- Throughtput
- Architecture
- Economics
Why we use distributed systems?

CAP Theorem
S. Gilbert and N. A. Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services.
SIGACT News, 33(2):51–59, 2002.

http://bravenewgeek.com/from-mainframe-to-microservice-an-introduction-to-distributed-systems/

Data Consistency?
Consensus Systems ,
Locking Services and
“barbaric“ algorithms

Data Consistency?
- Distributed Consensus : Costly and nearly impossible.
Multi Phase Commit, Stae Replication
- Two Phase Commit : Blocking, dependent on coordinator,
deadlocks inducing
- Three Phase commit : abort on timeouts, non blocking but easily
fails on network partitions

Data Consistency?

Data Consistency?
RAFT

Data Consistency?
Locking Services
Chubby (Google)
Zab (Yahoo)

@aphyr

“ Consistency is a property of your data,
not of your nodes”
@aphyr

AP Systems
Eventual Consistency

AP Systems
Eventual Consistency
- High Availability
- Low Latency
- Fault tolerant

Dynamo Systems
Riak
Voldemort
Cassandra

Conflicts resolution
- Semantic Resolution
- Vector Clocks
- Last write wins (LWW)

Vector ClocksLWW

Convergent Replicated
Data Types

CRDTs
- Take the consistency problem to the level of
Data Structures
- Their state resolves automatically (eventually) to
a single coherent value
- Maintain multiple copies of your data

CRDTs : Monotonicity
- Every new operation adds information
- Data is never immediately destroyed
- Most things are trasparent to the application

CRDTs : 2 types
- State based or convergent ( CvRDTs)
- Commutative or Operations based (CmRDTs)
*

CvRDTs
- All replicas connected
- At least once semantics usually
- State Changes advance upwards according to partial
order

CmRDTs
- All replicas connected
- Need Reliable broadcast with ordering guarantees
- Best suitable for commutative updates

CRDTs : Fancy words
CRDTs are idempotent, commutative monoids !!!

CRDTs : Fancy words
CRDTs are idempotent, commutative monoid !!!
aka
Joint Semi Lattice

- Idempotence : A * A = A
- Commutative : A*B = B* A
- Associative : A*(B*C) = (A*B)*C
Joint SemiLattice

CvRDTs

Some common CRDTs
- Registers (LWW, Multi Valued Register)
- G-Counter
- PN Counter
- G-Set, 2P-Set and OR-Set
- Graph
- Maps

G- Counter : grow only counter

PN-Counter
- Positive and Negative Counters
- Uses two G-counter
- One for increments(P) and another for
decrement(N)
- Result is the difference

G-Set
Grow only set, that only allows to add an element

2P-Set
- Two Phase Set
- Built with 2 G-set for add and removal
- Can’t add an already removed element :
Tombstone set
- Has a tombstone that maintains deleted
elements

LWW-Element-set
- Add Timestamp to “add”
and “remove” states wit h a
timestamp.
- Greatest timestamps wins
- Close to cassandra model

CRDTs in the Wild

www.jepsen.io
- Riak
- Cassandra
- Kafka
- etc.

Roshi
https://github.com/soundcloud/roshi

Eventuate

Limitations
- Garbage Collection
- Not always easy to have Semi Lattice for all
use case
- Inducing some strange behaviours
- Might require adding a stronger consistency
models: Raft, Paxos, etc
- Use when availability is really important
- Don’t use them for your billing application
- Definitely not a panacea

Eventual Consitency with CRDTS

More Related Content

Viewers also liked

Similar to Eventual Consitency with CRDTS

More from Samir Bessalah

Recently uploaded

Eventual Consitency with CRDTS

Editor's Notes