This document discusses different approaches to scaling Redis, including application sharding, using Envoy Proxy, and Redis Cluster. It provides an overview of each approach, demos of application sharding with a hash ring and Envoy Proxy with load balancing, and a demo of Redis Cluster. Pros and cons of each approach are also outlined. The document concludes with links to relevant resources and a Q&A section.

1. Scaling Redis
Application Sharding, Envoy Proxy, Redis Cluster

2. About me
● Dmtiry Polyakovsky, @dmitrypol
● http://dmitrypol.github.io/
● Sr. Software Developer at Oracle Cloud. Ideas are my own and not
necessarily of my employer.
● These slides - http://bit.ly/2TpKRfc
● Code samples - https://github.com/dmitrypol/redis_scale

3. Quick survey

5. Common Redis use-cases
1. Cache
2. Background job queue
3. API throttling
4. Feature flags
5. ...

6. Single Redis server

7. Read only replicas

8. Multiple Redis servers
Cache
Queue
API throttling
Feature flags

9. This talk is about
1. Application Sharding
2. Envoy Proxy
3. Redis Cluster

10. Application sharding

11. Demo
● HashRing
● worker

12. ● Easy to add servers
● Hard to move data
Scale out / in

13. Pros / cons
● Pros
○ Pre Redis 3.0 versions
○ Scale
○ No need for special clients
● Cons
○ Resharding
○ Complex biz logic

14. Questions about application sharding

16. Demo
● envoy.yaml
● web admin
● worker

17. Sharding
● MAGLEV lb_policy
○ 65537 table size
● HASH_RING lb_policy
● Hash tag {}
○ enable_hashtagging: true

18. ● Easy to add servers
● Hard to move data
Scale out / in

19. ● Supports all the single key commands
● Supports multi-key commands but keys must be on the same node
Supported / unsupported commands

20. Pros / cons
● Pros
○ Scale
○ Metrics
○ Pre Redis 3.0
○ No need for special client libraries
● Cons
○ Resharding
○ Additional layer
○ Single point

22. Questions about Envoy Proxy

23. Redis Cluster

25. ● Oracle Cloud BM.Standard2.52
● Cores: 104
● Memory: 768 GB
● Network: 2 x 25 Gbps
● List price per server - $3.3176 / hour
Cloud demo hardware

26. ● Currently in preview
● Managed Redis-aaS
● HA deployment via cross-AD with automatic failover
● Free trial at https://www.oracle.com/cloud/cloud-native/
Oracle Data Caching Cloud Service

27. Data processing pipeline

28. Pipeline with Redis Cluster

30. Demo
● redis.conf
● nodes.conf
● web dashboard
● worker

31. redis.conf
port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

32. nodes.conf
84cd… 127.0.0.1:7002@17002 master - 0 1553955105521 3 connected 10923-
16383
7e0dd1... 127.0.0.1:7001@17001 master - 0 1553955105000 2 connected 5461-
10922
4676b7... 127.0.0.1:7003@17003 slave
...
vars currentEpoch 6 lastVoteEpoch 0

33. ● Key level
● CRC16 of the key mod 16384
● Hash tags {}
Sharding

34. ● redis-cli
● Cache current node / slot mapping
Clients

35. ● Node ID is unique and unchangeable
● Default port + 10000
Cluster networking

36. ● Hash slot = group of keys
● Add / remove node from the cluster
Scaling / rebalancing

37. ● Supports all the single key commands
● Supports multi-key commands but keys must be on the same node
Supported / unsupported commands

38. Pros / cons
● Pros
○ Linear scale up to 1000
○ No need for proxy
○ Scale in / out
● Cons
○ Driver support
○ Only 1 Redis DB
○ May have small data loss in case of node failure

39. Questions about Redis Cluster

40. Links
● https://github.com/dmitrypol/redis_scale
● http://dmitrypol.github.io/categories.html#redis
● https://twitter.com/dmitrypol
● https://redis.io/topics/cluster-tutorial
● https://github.com/antirez/redis/blob/unstable/MANIFESTO
● https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/redis
● https://rdbtools.com/

41. Q & A