A presentation for a complete understanding of Redis, its setup types, and their complexity. How we can do a performance-based secure setup on the Kubernetes platform using the Redis Operator? https://github.com/OT-CONTAINER-KIT/redis-operator

1. Redis Scaling and
Observability

2. Agenda
➔ Redis Quiz
➔ Redis Overview
➔ Setup of Redis
➔ Scaling in Redis
➔ Observability of Redis

3. Redis Overview
Q1 - According to your understanding, what do you mean by Redis?
A. It’s a NoSQL database B. It’s a caching
system
C. It’s a SQL
database.
D. Both A and B
D. Both A and B

4. Redis Overview
Q2 - What are the different type of Redis setup are possible?
● Standalone setup
● Replicated cluster
setup
● Sharded cluster setup

5. Redis Overview
Q2 - Can we use Redis as a system with the traditional SQL databases?
A. True B. False
A. True

6. Redis Overview
In-Memory Database Data stored on system/server’s memory
Usage of Redis Often used as Cache to improve performance
Redis can also be used as full fledged database
as well.
Application Redis Database

7. Usage of Redis
Client Application Redis MySQL DB
1. Looks in cache system.
3. Cache the
new data
Cache hit simply
return the data.
2. Cache miss will look into the main
persistent database.

8. Key Properties of Redis
In-Memory Database Fast and Performant Schemaless

9. Types of Redis Setup
Sharded Cluster
Replicated Cluster
Standalone Standalone
Leader
Follower Follower
Leader Leader Leader
Follower Follower Follower

10. Standalone Setup
Redis
Server
Client
Applications
A single application or multiple
applications can connect to the
Redis server for writing and
reading keys.

11. Challenges of Standalone Setup
Out of Memory Data Bottleneck Failover Scalability

12. Replication
Leader
A B C D
Follower
A B C D
Follower
A B C D

13. Sharding
Leader 1
A B
Leader 2
C D
Leader 3
E F

14. Replication + Sharding
Leader 1
A B
Leader 2
C D
Leader 3
E F
Follower 1
A B
Follower 2
C D
Follower 3
E F
Replication of data from
leader to follower
Redirection of the
data

15. Replicated Setup
Redis
Leader
Client
Applications
Redis Follower
Redis Follower
Redis follower is replicating the
data that is being written on the
redis leader server
Client application will write the
data on the leader server of
Redis
All read requests will be redirected to the
followers node to reduce the network noise
from leader node of Redis.

16. Replicated Setup with Sentinel
Redis
Leader
Redis Follower
Redis Follower
Redis follower is replicating the
data that is being written on the
redis leader server
Redis
Sentinel
Redis sentinel keeping watch on
replicated cluster, if leader will fail, it will
promote the follower to the leader
System
Failure
Redis
Leader
Redis Follower

17. Challenges of Replicated Setup
Redis
Leader
Redis
Follower
Redis
Follower
● Dataset can become large
for leader and follower node
which can crash Redis
process.
● A single master can become
a bottleneck for the data

18. Sharded Cluster Setup
Leader 1 Leader 2 Leader 3
Follower 1 Follower 2 Folower 3
Leader is sharding data
between the leader
nodes
Followers replicating
data from leader nodes
Redirection for data
search and write
Write requests can be
sent on leader nodes
Read requests can be
sent on follower nodes
Client Application
Client Application

19. Setup Comparison
Standalone
Replicated
Setup
Sharded
Setup
Data
Bottleneck
High Medium Low
Out of
Memory Risk
High Medium Low

20. Setup Comparison
Standalone
Replicated
Setup
Sharded
Setup
No chances Medium
chances
High chances
Low Medium High
Failover
Scalability
Complexity
of Setup
Low Medium High

21. Scaling Redis Setup
Redis
Standalone
Redis
Leader
Redis
Follower
Redis
Follower
Leader 1 Leader 2 Leader 3
Follower 1 Follower 2 Folower 3
Replicated Setup
Sharded Setup

22. Scaling Replicated Setup
Redis
Leader
Redis Follower
Redis Follower
Redis Sentinel
Redis Follower
Redis Sentinel
Redis Sentinel
Scaling replication
followers
Scaling sentinel
for failover and
cluster

23. Scaling Sharded Setup
Redis Leader 1
A B
Redis Leader 2
C D
Redis Leader 3
F G
Redis Follower 1
A B
Redis Follower 2 Redis Follower 3
E H
C D E F G H
Redis Leader 4
Redis Follower 4
H
E
H
E
Resharding

24. Challenges in Scaling Redis
Sentinel configuration
and scaling
Resharding of data in
cluster mode
Complexity in scaling
Redis

25. Monitoring of Redis
Redis
Redis
Exporter
Prometheus
Grafana
Dashbaord
Prometheus pulling metrics
from the redis exporter at
defined interval
Grafana using
Prometheus as the
datasource for
metrics
Panels and dashboards
created on Grafana
platform

26. Important Monitoring Metrics for Redis
Uptime Latency Expired vs
non-expired
keys
Size of DB
CPU Memory Througput Connection
Count

27. Grafana Dashboard

28. Redis Setup on Kubernetes
Standalone
Redis Pod
Leader
Follower Follower
Leader
Follower
Leader Leader
Follower Follower

29. Challenges of Redis On Kubernetes
Dynamic IP update of
redis pod
Scaling Redis cluster on-
fly
Monitoring of Redis
system
Production ready
configuration
Security in terms of
Kubernetes and Redis
Managing persistent
data

30. Solution of Redis on Kubernetes
OT Redis Operator
Host, Run and Scale Redis cluster on
Kubernetes.

31. Features of Redis Operator
Multiple Redis version
supported
Standalone and Cluster
mode supported
Automated failover and
recovery is supported
In built monitoring
support with exporter
Password and
password-less support
TLS support for security
ipv4 ipv6
Ipv4 and Ipv6 support
for Redis
Detailed monitoring
dashboard of Grafana
On-fly upgrade without
downtime

32. Redis Operator Architecture

33. Standalone Setup of Redis

34. Cluster Setup of Redis

35. Enabling Monitoring
Redis Pod
Redis
Redis
Exporter

36. Upgrading Redis
v6 v7

37. TLS Integration

38. Contributors Behind Redis Operator
Abhishek Dubey,
Sr. DevOps Consultant,
Opstree Solutions
Sandeep Rawat,
CTO, Opstree Solutions
Shubham Gupta,
DevOps Intern, Opstree
Solutions
Mathieu Cesbron, Developer
from France
David Collam, Staff
Solutions Engineer,
Jetstack (Cert Manager)
Ivyan, Software Engineer,
DaoCloud

39. Contributors Behind Redis Operator
Andrew Thompson,
Software Engineer,
CrowdStrike
John, Software Engineer,
Naver
Hubert FONGARNAND,
Software Engineer, Ippon
Anish Chakraborty, Sr.
Software Engineer, Spotify
Tanuj Dwivedi, Sr. Software
Engineer, Dream11
Philippe Martin, Software
Engineer, RedHat

40. And Many More….

41. Thank you!
Contact:- abhishek.dubey@opstree.com, Sandeep@opstree.com
http://opstree.com
http://buildpiper.io