●
●
●
●
• Geo Distributed Trading/Bidding
- Auctions, Bids/Asks
Lists/Sorted Sets tracking Bids and Asks
• Dashboards & Scoreboards
- Tracking Geo Distributed Scoreboards
Sorted Sets tracking ordered scores
• Real-time session management and
Metering Apps
- HA for session management
- Tracking Usage/Consumption
- A/B testing
Hashes/Sets/Lists/Expiry Tracking Consumption Events
• Fraud Detection
- Geo Distributed Event Tracking
Sets Gathering Geo Distributed Events
• Social Engagement Apps
- Encoding Social Engagement
Distributed Counters for “Likes”, “Shares”, “Retweets”
• Collaboration Apps
- Constructing Smart Timelines
Merged Lists Ordering Posts
- Instant Messaging & Conversation Tracking
Merged Lists Ordering Conversations
●
● ⇒
●
● ⇒
● …
●
●
● inefficient
●
●
●
●
compressed and secured replication
compressed and secured
replication
compressed and secured replication
compressedand
securedreplication
conflict-free replicated data type ( data structure which
can be replicated across multiple computers in a network, each
holding local replication of the value (replica) where:
1. Local writes = Replicas can be updated independently and
concurrently WITHOUT coordination between the replicas
2. After replication, Inconsistency resolved = It is always
mathematically possible to resolve inconsistencies which might
occur
A
B
C
{A=1,B=0,C=0}
{A=1,B=1,C=0}
{A=0}
{B=0}
{C=0}
{A=0,B=0,C=1} {A=0,B=0,C=2}
{A=1,B=2,C=0}
{A=0,B=0,C=3}
{A=1,B=3,C=3}
{A=2,B=3,C=3}
●
●
●
●
15
Bi-directional secured
compressed
Bi-directional secured
compressed
Bi-directional
secured
compressed
CRDT
Module
CRDT
Module
CRDT
Module
●
●
<add image of the
items view>
Replica A Replica B
Two replicas, working over single data sets
Each step is done concurrently and then a synchronization takes
place
In reality, un-synced situations are limited to a few millisec
(depends on the network)
Examples are displayed from the client side point of view and not
the redis internals
Replica A Replica B
> INCRBY mykey 10
(integer) 10
> GET mykey
(nil)
Syncing
Synced
> GET mykey
“10”
> GET mykey
“10”
> GET mykey
“10”
Replica A Replica B
> INCRBY mykey 5
(integer) 15
> INCRBY mykey 8
(integer) 18
Syncing
Synced
> GET mykey
“23”
> GET mykey
“23”
Replica A Replica B
> DEL mykey
(integer) 1
> DECRBY mykey 3
(integer) 20
Syncing
Synced
> GET mykey
(nil)
> GET mykey
“-3”
> GET mykey
“-3”
Replica A Replica B
> DEL mykey
(integer)
> GET mykey
“-3”
Syncing
Synced
> GET mykey
(nil)
> GET mykey
(nil)
> GET mykey
(nil)
Replica A Replica B
> SADD myset X
(integer) 1
Syncing
Synced
> SMEMBERS myset
1) “X”
> SMEMBERS myset
1) “X”
> SMEMBERS myset
1) “X”
> SADD myset X
(integer) 1
> SREM myset X
(integer) 1
> SMEMBERS myset
(empty list or set)
Replica A Replica B
Syncing
Synced
> SMEMBERS myset
1) “Y”
2) “X”
> SMEMBERS myset
1) “Y”
> SMEMBERS myset
1) “Y”
> DEL myset
(integer) 0
> SMEMBERS myset
(empty list or set)
> SADD myset Y
(integer) 1
Replica A Replica B
> SET mykey “A”
OK
Syncing
Synced
> GET mykey
“A”
> GET mykey
1) “B”
> GET mykey
1) “B”
> GET mykey
“B”
> SET mykey “B”
OK
Replica B write time is later the replica A
Replica A Replica B
> DEL mykey
(integer)
> SET mykey “C”
OK
Syncing
Synced
> GET mykey
(nil)
> GET mykey
“C”
> GET mykey
“C”
> GET mykey
“C”
Strings
⇒ Observed Remove with Last write win
Counter
⇒ Observed Remove with local replica changes,
Sets
⇒ Observed Remove Set
Hashes
⇒ Based on the implementation of sets, with fields of registers and counters
Lists
⇒ Directed acyclic graph
Sorted Sets
⇒ Based on combining sets and Counters
RedisConf17 - Redis Labs - Multi-master Redis, A Deep Dive
RedisConf17 - Redis Labs - Multi-master Redis, A Deep Dive
RedisConf17 - Redis Labs - Multi-master Redis, A Deep Dive
RedisConf17 - Redis Labs - Multi-master Redis, A Deep Dive

RedisConf17 - Redis Labs - Multi-master Redis, A Deep Dive

  • 2.
  • 4.
    • Geo DistributedTrading/Bidding - Auctions, Bids/Asks Lists/Sorted Sets tracking Bids and Asks • Dashboards & Scoreboards - Tracking Geo Distributed Scoreboards Sorted Sets tracking ordered scores • Real-time session management and Metering Apps - HA for session management - Tracking Usage/Consumption - A/B testing Hashes/Sets/Lists/Expiry Tracking Consumption Events • Fraud Detection - Geo Distributed Event Tracking Sets Gathering Geo Distributed Events • Social Engagement Apps - Encoding Social Engagement Distributed Counters for “Likes”, “Shares”, “Retweets” • Collaboration Apps - Constructing Smart Timelines Merged Lists Ordering Posts - Instant Messaging & Conversation Tracking Merged Lists Ordering Conversations
  • 5.
  • 6.
  • 7.
  • 10.
    compressed and securedreplication compressed and secured replication compressed and secured replication compressedand securedreplication
  • 12.
    conflict-free replicated datatype ( data structure which can be replicated across multiple computers in a network, each holding local replication of the value (replica) where: 1. Local writes = Replicas can be updated independently and concurrently WITHOUT coordination between the replicas 2. After replication, Inconsistency resolved = It is always mathematically possible to resolve inconsistencies which might occur
  • 14.
  • 15.
  • 16.
  • 17.
  • 20.
    Replica A ReplicaB Two replicas, working over single data sets Each step is done concurrently and then a synchronization takes place In reality, un-synced situations are limited to a few millisec (depends on the network) Examples are displayed from the client side point of view and not the redis internals
  • 22.
    Replica A ReplicaB > INCRBY mykey 10 (integer) 10 > GET mykey (nil) Syncing Synced > GET mykey “10” > GET mykey “10” > GET mykey “10”
  • 23.
    Replica A ReplicaB > INCRBY mykey 5 (integer) 15 > INCRBY mykey 8 (integer) 18 Syncing Synced > GET mykey “23” > GET mykey “23”
  • 24.
    Replica A ReplicaB > DEL mykey (integer) 1 > DECRBY mykey 3 (integer) 20 Syncing Synced > GET mykey (nil) > GET mykey “-3” > GET mykey “-3”
  • 25.
    Replica A ReplicaB > DEL mykey (integer) > GET mykey “-3” Syncing Synced > GET mykey (nil) > GET mykey (nil) > GET mykey (nil)
  • 27.
    Replica A ReplicaB > SADD myset X (integer) 1 Syncing Synced > SMEMBERS myset 1) “X” > SMEMBERS myset 1) “X” > SMEMBERS myset 1) “X” > SADD myset X (integer) 1 > SREM myset X (integer) 1 > SMEMBERS myset (empty list or set)
  • 28.
    Replica A ReplicaB Syncing Synced > SMEMBERS myset 1) “Y” 2) “X” > SMEMBERS myset 1) “Y” > SMEMBERS myset 1) “Y” > DEL myset (integer) 0 > SMEMBERS myset (empty list or set) > SADD myset Y (integer) 1
  • 30.
    Replica A ReplicaB > SET mykey “A” OK Syncing Synced > GET mykey “A” > GET mykey 1) “B” > GET mykey 1) “B” > GET mykey “B” > SET mykey “B” OK Replica B write time is later the replica A
  • 31.
    Replica A ReplicaB > DEL mykey (integer) > SET mykey “C” OK Syncing Synced > GET mykey (nil) > GET mykey “C” > GET mykey “C” > GET mykey “C”
  • 32.
    Strings ⇒ Observed Removewith Last write win Counter ⇒ Observed Remove with local replica changes, Sets ⇒ Observed Remove Set Hashes ⇒ Based on the implementation of sets, with fields of registers and counters Lists ⇒ Directed acyclic graph Sorted Sets ⇒ Based on combining sets and Counters