SlideShare a Scribd company logo
1 of 31
Techniques for Synchronizing
In-Memory Caches with Redis
Ben Malec, Paylocity
Introduction and a Bit of Background
Simple Web Farm
Web Server #1
SQL
database
Web Server #2 Web Server #3 Web Server #4
Data access frequency
Datachangefrequency
Rarely changed,
rarely accessed
Often changing,
often accessed
Rarely changing,
often accessed
Often changing,
rarely accessed
Cacheability Quadrants
Web Server #1
SQL
database
in-process
cache
Web Server #2
in-process
cache
Web Server #3
in-process
cache
Web Server #4
in-process
cache
Web Farm with In-Process Caches
Cache Data Lag
Data stored in the in-process caches lags behind the source of
truth.
• Bad user experience
• Usual solution is to shorter cache expiration times, but it’s
just a trade-off
• Doesn’t eliminate the problem, only reduces the length of the lag
• Shorter cache times mean more database hits
• What would be great is push notification from the source of
truth
• But it’s not straightforward to implement push notifications from a
SQL backend
• Easy to flood the network with sync traffic
Cross-Server Cache Data Inconsistency
Over time the absolute expiration time of any specific key falls
out of sync, meaning the data changes depending on which
box serves the request.
• Even worse user experience
• Another non-trivial issue to solve to solve
• Need to implement two-way communication between all
nodes
• Difficult to resolve who wins when multiple nodes update
the same value at the same time
• Easy to flood network with sync messages
Cache Stampede
If every server has its own copy of cached data, then every
server needs to refresh it, too
• Bad at process start-up, or if multiple servers have close
expiration times
• Really bad during pooled deploys when an entire pool comes
up
• Problem continues to grow as the number of servers
increases
• Easy to overload back end with requests (are you noticing a
trend here?)
One Solution: Shared Redis Cache
Web Server #1 Web Server #2 Web Server #3 Web Server #4
SQL
database
Shared Redis cache
Classic Redis use case with lots of advantages:
• Solves the data consistency problem completely.
• Reduces cache data lag with a write-through cache
implementation (but watch out for DBAs with ad-hoc scripts!)
• No more cache stampede at the database (only need to
update Redis once, regardless how many clients)
But…
• Now we have a TCP roundtrip per cache access
• While Redis is incredibly fast, local RAM access is still many
times faster than network I/O and a deserialization step
Web Server #1
SQL
database
in-process
cache
Web Server #2
in-process
cache
Web Server #3
in-process
cache
Web Server #4
in-process
cache
Kevin Says, “What if…”
Oh, and By the Way…
1. Solve the data consistency issue
2. Eliminate any data lag between the in-process caches and
Redis
3. Don’t blow up the network!
We haven’t been able to solve these problems effectively in the
past, but what about now that Redis is part of the infrastructure?
Slowly the Pieces Fall Into Place…
First piece to fall in place is Redis Pub/Sub for inter-server
communications
• Trivial to implement
• Just plain works
• Can use to both synchronize nodes (maintain consistency) and
push changes (minimize lag) to the client.
• But what data to send without saturating the network?
Approach #1: Broadcast all data changes to all nodes
• Yes, you will blow up the network
• Low efficiency, all nodes receive changed data even if they’ll
never use it.
• Lots of challenges around ensuring all nodes have identical
data when multiple nodes update the same key at near-
identical times.
Solving the Data Consistency Problem
Approach #2: Broadcast the key that changed to all nodes
• Less network traffic than sending key/value pairs
• Solves the consistency issue of broadcasting values because
we’re just telling nodes to hit Redis the next time the key is
accessed.
• But Redis’ flexibility works against us here: keys can be up to
512MB, opening up the possibility of blowing up the network
just broadcasting keys.
Solving the Data Consistency Problem
Approach #3: Instead of broadcasting keys to all nodes, why not
partition keys into 16,384 buckets and just broadcast the 16-bit
bucket ID?
• Inspired by Redis cluster hash slot implementation
• Short, fixed size synchronization messages, regardless of key
size
• No value synchronization issues, just tell each client to hit
Redis next time a key matching the hash slot is requested.
• Now, just need to implement ;-)
Solving the Data Consistency Problem
Still, there’s a lot of things to solve:
• Since we’ve grouped keys together by their hash slot, when
we need to evict a key we actually will evict all keys sharing
the same hash slot
• An obvious solution would be to evict all values from the
local cache whose key falls in the same hash slot
• But that’s not practical, would have to scan all cache keys and
calculate their hash slot
Implementation Challenges
The approach Paylocity arrived at has three main features:
• A dictionary of hash slots and the timestamp when a key in
that hash slot was updated (the lastUpdated dictionary)
• Items written to the in-process cache include the key’s hash
slot and the timestamp when the object written to the in-
process cache
• Whenever a value is updated, a sync message containing the
updated hash slot is published via Redis pub/sub
Paylocity’s Solution
Key Value
HashSlot 14587
Timestamp 150938476
Value <object>
App:Employee:1736
HashSlot 1228
Timestamp 163827634
Value <object>
App:Employee:2367
HashSlot 9036
Timestamp 180985776
Value <object>
App:Employee:3123
HashSlot 1231
Timestamp 179872198
Value <object>
App:Employee:4273
In-Process Cache
lastUpdated Dictionary
Hashslot Timestamp
173658476
163827634
163928374
180028372
1227
1228
1229
1230
⋮⋮
⋮⋮
Paylocity’s Solution
Additionally, a Redis pub/sub message hander listens for
synchronization messages
• Whenever a sync message is received, the hash slot entry in the
lastUpdated dictionary will be updated with the current
timestamp
• When retrieving data from the in-process cache, compare the
timestamp in the cache entry with the timestamp in the
lastUpdated dictionary. If the lastUpdated timestamp is greater
than the cache entry, the entry is out-of-date and should be
discarded.
Here’s the flow in the end:
More Implementation Details
Reading a value from the cache
Read entry from
in-process cache
Read hash slot timetamp
from lastUpdated dictionary
Update timestamp in
lastUpdated dictionary
Write entry to
in-process cache
Return value
to client
Calculate key
hash slot
Does key
exist in the
in-process
cache?
Is lastUpdated
dictionary
timestamp
greater than
the cache entry
timestamp?
Read value
from Redis
Yes
No
Yes
No
Adding a value to the cache
Calculate key
hash slot
Get current
timestamp
Add timestamp to
lastUpdated dictionary
Write entry (with
timestamp and hash slot)
to the in-process cache
Write key/value to Redis
Publish update message
to all clients
Hash slot
exists in the
lastUpdated
dictionary?
No
Yes
Most scenarios are solved by broadcasting only keys and using
Redis as a single source of truth for the cache, but not all:
• Trickiest situation occurs when a node receives a key
invalidation message just after it writes to Redis. Who actually
won?
• Must prevent state where a local node believes it has the
correct data in its in-process cache, but actually doesn’t
• Could implement a master clock, but that introduces a
bottleneck as well as a single point of failure
• Could use a distributed lock algorithm like Redlock
Still Some Timing Issues Remain
Timing Challenges
ConcurrencyConsistency
Instead of distributed locks or a master clock, exploit order-of-
operation
• Leverage the fact that Redis is the source of truth for this cache
• Deceptively simple, high concurrency
• Update Redis, then publish the sync message
• No possibility of a client being notified before the Redis value was updated
• Always grab the current timestamp before writing to Redis, the in-
process cache, or the lastUpdated dictionary
• Eliminate the possibility that we store a timestamp that’s more recent than
the actual time we wrote the value
Still, one corner case exists…
Order of Operation
RedisMultilevelCacheClient app Stopwatch
Timestamp increments
Add(key, value)
GetTimestamp()
HandleSyncMessage()
UpdateRedis, in-process cache, expiration dictionary
Timestamp increments
One Last Corner Case…
Underlying problem is that we’re using an incrementing timestamp to
determine the order operations occurred
• Can’t measure something smaller than the resolution of the measuring
device!
• Easier to visualize if you imagine the timer resolution to be a minute
• In practice, not an issue because it would require a Redis write, sync
message publish, and sync message handling within ~300 nanoseconds
• But still, don’t want to leave known issues open
In the end, just need to subtract 1 from timestamp obtained at the start of
the operation (!!!)
• This effectively forces the client to immediately re-read the value from
Redis in cases where the timer resolution prevents us from actually
determine which operation occurred first
…With an Ultimately Simple Solution
• Early testing revealed many more hits to Redis than predicted
• Root cause was that the clients were processing the update pub/sub
message they published, causing them to re-read the value they just
wrote to Redis
• Solution was to have the pub/sub handler ignore messages that
originated from the same cache provider instance
• Updating Redis consists of executing two Redis commands,
one to update the value, and the second to notify other clients
of the change
• But we don’t want to incur two TCP roundtrip.
• Lua scripting to the rescue!
And a Few Final Optimizations
Unresolved Concerns and Future Plans
• Potential for cache thrashing since groups of keys are evicted
by hash slot
• So far no issues at Paylocity with this
• Could expand the number of hash slots to better separate keys
• Current don’t handle a lost sync message well
• Essentially works an independent in-process cache
• Leverage Redis Pub/Sub to collect and publish client hit/miss
metrics
• Not too difficult to get this data into the ELK stack
• Implement XFETCH to optimize cache reload
• Support more Redis data types!
• Example implementation:
https://github.com/bmalec/RedisMultilevelCache
• Questions?
That’s All, Folks!
Thank you

More Related Content

What's hot

Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture ForumChristopher Spring
 
HBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBaseHBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBaseenissoz
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcachedJurriaan Persyn
 
HBase in Practice
HBase in PracticeHBase in Practice
HBase in Practicelarsgeorge
 
Grokking TechTalk #33: High Concurrency Architecture at TIKI
Grokking TechTalk #33: High Concurrency Architecture at TIKIGrokking TechTalk #33: High Concurrency Architecture at TIKI
Grokking TechTalk #33: High Concurrency Architecture at TIKIGrokking VN
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisDvir Volk
 
Kafka: All an engineer needs to know
Kafka: All an engineer needs to knowKafka: All an engineer needs to know
Kafka: All an engineer needs to knowThao Huynh Quang
 
patroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymentpatroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymenthyeongchae lee
 
Learning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
Learning Rust the Hard Way for a Production Kafka + ScyllaDB PipelineLearning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
Learning Rust the Hard Way for a Production Kafka + ScyllaDB PipelineScyllaDB
 
From Mainframe to Microservice: An Introduction to Distributed Systems
From Mainframe to Microservice: An Introduction to Distributed SystemsFrom Mainframe to Microservice: An Introduction to Distributed Systems
From Mainframe to Microservice: An Introduction to Distributed SystemsTyler Treat
 
APACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka StreamsAPACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka StreamsKetan Gote
 
A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redisZhichao Liang
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm Chandler Huang
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...Altinity Ltd
 
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATSDeploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATSNATS
 
Streaming architecture patterns
Streaming architecture patternsStreaming architecture patterns
Streaming architecture patternshadooparchbook
 
DRP (Stretch Cluster) for HDP - Future of Data : Paris
DRP (Stretch Cluster) for HDP - Future of Data : Paris DRP (Stretch Cluster) for HDP - Future of Data : Paris
DRP (Stretch Cluster) for HDP - Future of Data : Paris Mohamed Mehdi Ben Aissa
 
Redis in Practice
Redis in PracticeRedis in Practice
Redis in PracticeNoah Davis
 
All about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdfAll about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdfAltinity Ltd
 
Redis cluster
Redis clusterRedis cluster
Redis clusteriammutex
 

What's hot (20)

Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture Forum
 
HBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBaseHBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBase
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
 
HBase in Practice
HBase in PracticeHBase in Practice
HBase in Practice
 
Grokking TechTalk #33: High Concurrency Architecture at TIKI
Grokking TechTalk #33: High Concurrency Architecture at TIKIGrokking TechTalk #33: High Concurrency Architecture at TIKI
Grokking TechTalk #33: High Concurrency Architecture at TIKI
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Kafka: All an engineer needs to know
Kafka: All an engineer needs to knowKafka: All an engineer needs to know
Kafka: All an engineer needs to know
 
patroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymentpatroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deployment
 
Learning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
Learning Rust the Hard Way for a Production Kafka + ScyllaDB PipelineLearning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
Learning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
 
From Mainframe to Microservice: An Introduction to Distributed Systems
From Mainframe to Microservice: An Introduction to Distributed SystemsFrom Mainframe to Microservice: An Introduction to Distributed Systems
From Mainframe to Microservice: An Introduction to Distributed Systems
 
APACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka StreamsAPACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka Streams
 
A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redis
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
 
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATSDeploy Secure and Scalable Services Across Kubernetes Clusters with NATS
Deploy Secure and Scalable Services Across Kubernetes Clusters with NATS
 
Streaming architecture patterns
Streaming architecture patternsStreaming architecture patterns
Streaming architecture patterns
 
DRP (Stretch Cluster) for HDP - Future of Data : Paris
DRP (Stretch Cluster) for HDP - Future of Data : Paris DRP (Stretch Cluster) for HDP - Future of Data : Paris
DRP (Stretch Cluster) for HDP - Future of Data : Paris
 
Redis in Practice
Redis in PracticeRedis in Practice
Redis in Practice
 
All about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdfAll about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdf
 
Redis cluster
Redis clusterRedis cluster
Redis cluster
 

Similar to RedisConf18 - Techniques for Synchronizing In-Memory Caches with Redis

Donatas Mažionis, Building low latency web APIs
Donatas Mažionis, Building low latency web APIsDonatas Mažionis, Building low latency web APIs
Donatas Mažionis, Building low latency web APIsTanya Denisyuk
 
FreeSWITCH as a Microservice
FreeSWITCH as a MicroserviceFreeSWITCH as a Microservice
FreeSWITCH as a MicroserviceEvan McGee
 
Architecting for the cloud elasticity security
Architecting for the cloud elasticity securityArchitecting for the cloud elasticity security
Architecting for the cloud elasticity securityLen Bass
 
Resilient Predictive Data Pipelines (GOTO Chicago 2016)
Resilient Predictive Data Pipelines (GOTO Chicago 2016)Resilient Predictive Data Pipelines (GOTO Chicago 2016)
Resilient Predictive Data Pipelines (GOTO Chicago 2016)Sid Anand
 
Membase Intro from Membase Meetup San Francisco
Membase Intro from Membase Meetup San FranciscoMembase Intro from Membase Meetup San Francisco
Membase Intro from Membase Meetup San FranciscoMembase
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitterRoger Xia
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...smallerror
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...xlight
 
HBase Low Latency, StrataNYC 2014
HBase Low Latency, StrataNYC 2014HBase Low Latency, StrataNYC 2014
HBase Low Latency, StrataNYC 2014Nick Dimiduk
 
HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)
HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)
HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)Ontico
 
Openstack meetup lyon_2017-09-28
Openstack meetup lyon_2017-09-28Openstack meetup lyon_2017-09-28
Openstack meetup lyon_2017-09-28Xavier Lucas
 
Building Big Data Streaming Architectures
Building Big Data Streaming ArchitecturesBuilding Big Data Streaming Architectures
Building Big Data Streaming ArchitecturesDavid Martínez Rego
 
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...DataStax
 

Similar to RedisConf18 - Techniques for Synchronizing In-Memory Caches with Redis (20)

Redis
RedisRedis
Redis
 
Donatas Mažionis, Building low latency web APIs
Donatas Mažionis, Building low latency web APIsDonatas Mažionis, Building low latency web APIs
Donatas Mažionis, Building low latency web APIs
 
FreeSWITCH as a Microservice
FreeSWITCH as a MicroserviceFreeSWITCH as a Microservice
FreeSWITCH as a Microservice
 
Architecting for the cloud elasticity security
Architecting for the cloud elasticity securityArchitecting for the cloud elasticity security
Architecting for the cloud elasticity security
 
Resilient Predictive Data Pipelines (GOTO Chicago 2016)
Resilient Predictive Data Pipelines (GOTO Chicago 2016)Resilient Predictive Data Pipelines (GOTO Chicago 2016)
Resilient Predictive Data Pipelines (GOTO Chicago 2016)
 
Membase Intro from Membase Meetup San Francisco
Membase Intro from Membase Meetup San FranciscoMembase Intro from Membase Meetup San Francisco
Membase Intro from Membase Meetup San Francisco
 
Scaling tappsi
Scaling tappsiScaling tappsi
Scaling tappsi
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
 
Fixing_Twitter
Fixing_TwitterFixing_Twitter
Fixing_Twitter
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
HBase Low Latency, StrataNYC 2014
HBase Low Latency, StrataNYC 2014HBase Low Latency, StrataNYC 2014
HBase Low Latency, StrataNYC 2014
 
HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)
HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)
HighLoad Solutions On MySQL / Xiaobin Lin (Alibaba)
 
Openstack meetup lyon_2017-09-28
Openstack meetup lyon_2017-09-28Openstack meetup lyon_2017-09-28
Openstack meetup lyon_2017-09-28
 
Building Big Data Streaming Architectures
Building Big Data Streaming ArchitecturesBuilding Big Data Streaming Architectures
Building Big Data Streaming Architectures
 
No sql exploration keyvaluestore
No sql exploration   keyvaluestoreNo sql exploration   keyvaluestore
No sql exploration keyvaluestore
 
Real time database
Real time databaseReal time database
Real time database
 
CoAP Talk
CoAP TalkCoAP Talk
CoAP Talk
 
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...
 
Redis - Partitioning
Redis - PartitioningRedis - Partitioning
Redis - Partitioning
 

More from Redis Labs

Redis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Labs
 
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs
 
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs
 
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs
 
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs
 
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs
 
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Redis Labs
 
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Redis Labs
 
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...Redis Labs
 
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Redis Labs
 
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Redis Labs
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs
 
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020Redis Labs
 
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Redis Labs
 
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Redis Labs
 
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Redis Labs
 

More from Redis Labs (20)

Redis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redis
 
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
 
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
 
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
 
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
 
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
 
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
 
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
 
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
 
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
 
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
 
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
 
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
 
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
 
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
 

Recently uploaded

Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseWSO2
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxMarkSteadman7
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Bhuvaneswari Subramani
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewDianaGray10
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهMohamed Sweelam
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...caitlingebhard1
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Zilliz
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxFIDO Alliance
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....rightmanforbloodline
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data SciencePaolo Missier
 

Recently uploaded (20)

Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern Enterprise
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 

RedisConf18 - Techniques for Synchronizing In-Memory Caches with Redis

  • 1. Techniques for Synchronizing In-Memory Caches with Redis Ben Malec, Paylocity
  • 2. Introduction and a Bit of Background
  • 3. Simple Web Farm Web Server #1 SQL database Web Server #2 Web Server #3 Web Server #4
  • 4. Data access frequency Datachangefrequency Rarely changed, rarely accessed Often changing, often accessed Rarely changing, often accessed Often changing, rarely accessed Cacheability Quadrants
  • 5. Web Server #1 SQL database in-process cache Web Server #2 in-process cache Web Server #3 in-process cache Web Server #4 in-process cache Web Farm with In-Process Caches
  • 6. Cache Data Lag Data stored in the in-process caches lags behind the source of truth. • Bad user experience • Usual solution is to shorter cache expiration times, but it’s just a trade-off • Doesn’t eliminate the problem, only reduces the length of the lag • Shorter cache times mean more database hits • What would be great is push notification from the source of truth • But it’s not straightforward to implement push notifications from a SQL backend • Easy to flood the network with sync traffic
  • 7. Cross-Server Cache Data Inconsistency Over time the absolute expiration time of any specific key falls out of sync, meaning the data changes depending on which box serves the request. • Even worse user experience • Another non-trivial issue to solve to solve • Need to implement two-way communication between all nodes • Difficult to resolve who wins when multiple nodes update the same value at the same time • Easy to flood network with sync messages
  • 8. Cache Stampede If every server has its own copy of cached data, then every server needs to refresh it, too • Bad at process start-up, or if multiple servers have close expiration times • Really bad during pooled deploys when an entire pool comes up • Problem continues to grow as the number of servers increases • Easy to overload back end with requests (are you noticing a trend here?)
  • 9. One Solution: Shared Redis Cache Web Server #1 Web Server #2 Web Server #3 Web Server #4 SQL database
  • 10. Shared Redis cache Classic Redis use case with lots of advantages: • Solves the data consistency problem completely. • Reduces cache data lag with a write-through cache implementation (but watch out for DBAs with ad-hoc scripts!) • No more cache stampede at the database (only need to update Redis once, regardless how many clients) But… • Now we have a TCP roundtrip per cache access • While Redis is incredibly fast, local RAM access is still many times faster than network I/O and a deserialization step
  • 11. Web Server #1 SQL database in-process cache Web Server #2 in-process cache Web Server #3 in-process cache Web Server #4 in-process cache Kevin Says, “What if…”
  • 12. Oh, and By the Way… 1. Solve the data consistency issue 2. Eliminate any data lag between the in-process caches and Redis 3. Don’t blow up the network! We haven’t been able to solve these problems effectively in the past, but what about now that Redis is part of the infrastructure?
  • 13. Slowly the Pieces Fall Into Place… First piece to fall in place is Redis Pub/Sub for inter-server communications • Trivial to implement • Just plain works • Can use to both synchronize nodes (maintain consistency) and push changes (minimize lag) to the client. • But what data to send without saturating the network?
  • 14. Approach #1: Broadcast all data changes to all nodes • Yes, you will blow up the network • Low efficiency, all nodes receive changed data even if they’ll never use it. • Lots of challenges around ensuring all nodes have identical data when multiple nodes update the same key at near- identical times. Solving the Data Consistency Problem
  • 15. Approach #2: Broadcast the key that changed to all nodes • Less network traffic than sending key/value pairs • Solves the consistency issue of broadcasting values because we’re just telling nodes to hit Redis the next time the key is accessed. • But Redis’ flexibility works against us here: keys can be up to 512MB, opening up the possibility of blowing up the network just broadcasting keys. Solving the Data Consistency Problem
  • 16. Approach #3: Instead of broadcasting keys to all nodes, why not partition keys into 16,384 buckets and just broadcast the 16-bit bucket ID? • Inspired by Redis cluster hash slot implementation • Short, fixed size synchronization messages, regardless of key size • No value synchronization issues, just tell each client to hit Redis next time a key matching the hash slot is requested. • Now, just need to implement ;-) Solving the Data Consistency Problem
  • 17. Still, there’s a lot of things to solve: • Since we’ve grouped keys together by their hash slot, when we need to evict a key we actually will evict all keys sharing the same hash slot • An obvious solution would be to evict all values from the local cache whose key falls in the same hash slot • But that’s not practical, would have to scan all cache keys and calculate their hash slot Implementation Challenges
  • 18. The approach Paylocity arrived at has three main features: • A dictionary of hash slots and the timestamp when a key in that hash slot was updated (the lastUpdated dictionary) • Items written to the in-process cache include the key’s hash slot and the timestamp when the object written to the in- process cache • Whenever a value is updated, a sync message containing the updated hash slot is published via Redis pub/sub Paylocity’s Solution
  • 19. Key Value HashSlot 14587 Timestamp 150938476 Value <object> App:Employee:1736 HashSlot 1228 Timestamp 163827634 Value <object> App:Employee:2367 HashSlot 9036 Timestamp 180985776 Value <object> App:Employee:3123 HashSlot 1231 Timestamp 179872198 Value <object> App:Employee:4273 In-Process Cache lastUpdated Dictionary Hashslot Timestamp 173658476 163827634 163928374 180028372 1227 1228 1229 1230 ⋮⋮ ⋮⋮ Paylocity’s Solution
  • 20. Additionally, a Redis pub/sub message hander listens for synchronization messages • Whenever a sync message is received, the hash slot entry in the lastUpdated dictionary will be updated with the current timestamp • When retrieving data from the in-process cache, compare the timestamp in the cache entry with the timestamp in the lastUpdated dictionary. If the lastUpdated timestamp is greater than the cache entry, the entry is out-of-date and should be discarded. Here’s the flow in the end: More Implementation Details
  • 21. Reading a value from the cache Read entry from in-process cache Read hash slot timetamp from lastUpdated dictionary Update timestamp in lastUpdated dictionary Write entry to in-process cache Return value to client Calculate key hash slot Does key exist in the in-process cache? Is lastUpdated dictionary timestamp greater than the cache entry timestamp? Read value from Redis Yes No Yes No
  • 22. Adding a value to the cache Calculate key hash slot Get current timestamp Add timestamp to lastUpdated dictionary Write entry (with timestamp and hash slot) to the in-process cache Write key/value to Redis Publish update message to all clients Hash slot exists in the lastUpdated dictionary? No Yes
  • 23. Most scenarios are solved by broadcasting only keys and using Redis as a single source of truth for the cache, but not all: • Trickiest situation occurs when a node receives a key invalidation message just after it writes to Redis. Who actually won? • Must prevent state where a local node believes it has the correct data in its in-process cache, but actually doesn’t • Could implement a master clock, but that introduces a bottleneck as well as a single point of failure • Could use a distributed lock algorithm like Redlock Still Some Timing Issues Remain
  • 25. Instead of distributed locks or a master clock, exploit order-of- operation • Leverage the fact that Redis is the source of truth for this cache • Deceptively simple, high concurrency • Update Redis, then publish the sync message • No possibility of a client being notified before the Redis value was updated • Always grab the current timestamp before writing to Redis, the in- process cache, or the lastUpdated dictionary • Eliminate the possibility that we store a timestamp that’s more recent than the actual time we wrote the value Still, one corner case exists… Order of Operation
  • 26. RedisMultilevelCacheClient app Stopwatch Timestamp increments Add(key, value) GetTimestamp() HandleSyncMessage() UpdateRedis, in-process cache, expiration dictionary Timestamp increments One Last Corner Case…
  • 27. Underlying problem is that we’re using an incrementing timestamp to determine the order operations occurred • Can’t measure something smaller than the resolution of the measuring device! • Easier to visualize if you imagine the timer resolution to be a minute • In practice, not an issue because it would require a Redis write, sync message publish, and sync message handling within ~300 nanoseconds • But still, don’t want to leave known issues open In the end, just need to subtract 1 from timestamp obtained at the start of the operation (!!!) • This effectively forces the client to immediately re-read the value from Redis in cases where the timer resolution prevents us from actually determine which operation occurred first …With an Ultimately Simple Solution
  • 28. • Early testing revealed many more hits to Redis than predicted • Root cause was that the clients were processing the update pub/sub message they published, causing them to re-read the value they just wrote to Redis • Solution was to have the pub/sub handler ignore messages that originated from the same cache provider instance • Updating Redis consists of executing two Redis commands, one to update the value, and the second to notify other clients of the change • But we don’t want to incur two TCP roundtrip. • Lua scripting to the rescue! And a Few Final Optimizations
  • 29. Unresolved Concerns and Future Plans • Potential for cache thrashing since groups of keys are evicted by hash slot • So far no issues at Paylocity with this • Could expand the number of hash slots to better separate keys • Current don’t handle a lost sync message well • Essentially works an independent in-process cache • Leverage Redis Pub/Sub to collect and publish client hit/miss metrics • Not too difficult to get this data into the ELK stack • Implement XFETCH to optimize cache reload • Support more Redis data types!