This presentation discusses unconventional uses of Redis, including as an API for transient data where client processes coordinate through short-lived Redis instances, for weakly synchronized volatile data using Redis replication, as a parameter server in ML applications, and as a dynamically scaling cache associated with worker processes. The presenter notes benefits of Redis like being understandable, modifiable, having implemented useful network and data structure components, being lightweight and fast, and allowing custom logic through modules. Modifications made to Redis are also discussed.

1. PRESENTED BY

2. PRESENTED BY
• This presentation reflects the analysis and views of Mauro Calderara. No recipient
should interpret this presentation to represent the general views of Citadel or its
personnel. Facts, analysis, and views presented in this presentation have not been
reviewed by, and may not reflect information known to, other Citadel professionals.
• Assumptions, opinions, views, and estimates constitute Mr. Calderara’s judgment as
of the date given and are subject to change without notice and without any duty to
update. Citadel is not responsible for any errors or omissions contained in this
presentation and accepts no liability whatsoever for any direct or consequential
loss arising from your use of this presentation or its contents.
• This document and the information it contains may not be disclosed to any persons
other than those for whom it is intended, nor should this document or the
information it contains be copied, distributed, or redistributed, in whole or in part,
without the prior written consent of Citadel.
• Any castle trademarks, service marks and logos used in this document are
trademarks or service marks or registered trademarks or service marks of Citadel.

3. PRESENTED BY
Show unconventional use patterns to
provide a different angle on redis
Main idea of this talk
Show unconventional* use patterns to
provide a different angle on redis
Show unconventional* use patterns to
provide a different angle** on redis

4. PRESENTED BY
Use case 1: Using redis as an API to transient data
Proprietary
process
Standalone
redis
• Redis lifetime bound to processes life-time, relatively short-lived
• Clients coordinate via the redis instances
• Sharding, erasure coding, fail over, load balancing, eviction handled client-side

5. PRESENTED BY
Use case 2: Using redis for weakly synchronized volatile data
Proprietary
process
Standalone
redis
• Use redis’ implement of reliable, eventually consistent replication*
• Very lightweight solution
• Semantics are very clear and explicit

6. PRESENTED BY
Use case 3: Using redis as a parameter server in ML applications
Proprietary
process
Standalone
redis
• Coordinate embarrassingly parallel / map-reduce like computations
• Can do advanced math on the parameter server using redis modules

7. PRESENTED BY
Use case 4: Using redis as a dynamically scaling cache
Proprietary
process
Standalone
redis
• redis associated with each worker contributing bandwidth to cache
• Coordination via centralized redis
• Fixed ratio of workers/cache → horizontal scalability
• Can push parts of the application logic into the cache layer using modules!

8. PRESENTED BY
Why use redis for this?
• I can understand it
• I can modify it
• Redis has implemented a very nice box of network, scheduling, data structures,
persistence, replication, expiration parts that are generic → quick to deliver
• Very lightweight and fast
• Can embed complicate logic through writing your own modules
• Very useful for generic process coordination through LUA scripts

9. PRESENTED BY
Modifications we did
• ACLs (going to be in mainline redis)
• Modules (now in mainline redis)
• Optimizations for bulk data in server and client
• Multi-threaded I/O (going to be in mainline redis but differently)
• Other, minor things
• Considering: redis as a library / embedded redis

10. PRESENTED BY
Takeaway
One can also think of redis as more of a protocol
and library than a service
• Great that it is so lightweight
• Can often stay away from complex architectures
• Hard to overestimate the impact of small, well written, modifiable code base

11. Thank you!
Thank you!