Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
REDIS IN A MULTI-TENANT
ENVIRONMENT
Iccha Sethi, Software Developer
Twitter: @IcchaSethi
2
IcchaSethi.__dict__
{
“Name”: “Iccha Sethi”,
“Occupation”: “Software Developer”,
“Team”: “Redis”,
“Company”: “Object Roc...
Why does High Availability matter for Redis?
3
Why does High Availability matter for Redis?
4
Outline
• Architecture
• Security and Isolation
• Know thy environment
5
Outline
• Architecture
• Security and Isolation
• Know thy environment
6
7
Architecture
Outline
8
Redis
Master
Access HA Redis:
Dns: read/write port 6379
password
Outline
9
Redis
Master
Redis
Slave
Access HA Redis:
DNS: read/write port 6379
password
Outline
10
VIP
LB1 (Master)
HAProxy
Redis
Master
Redis
Slave
Access HA Redis:
DNS: read/write
port 6379
password
Outline
11
VIP
LB1 (Master)
HAProxy
LB2
(Backup)
HAProxy
keepalived
Redis
Master
Redis
Slave
Access HA Redis:
DNS: read/wr...
12
Sentinels
13
Highly Available REDIS
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Slave
KeepAliveD
H...
14
Architecture - Opportunities
Opportunities
•  Failover is easy
•  Easy to introduce changes
•  Reusable architecture
15
Opportunities
•  Failover is easy
•  Easy to introduce changes
•  Reusable architecture
16
17
Fail Over – Redis Nodes
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Slave
KeepAliveD
...
18
Fail Over – Redis Nodes
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Slave
KeepAliveD
...
19
Fail Over – Redis Nodes
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Master
KeepAliveD...
20
Fail Over – Redis Nodes
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Master
KeepAliveD...
21
Fail Over – Redis Nodes
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Master
KeepAliveD...
22
Fail Over – Redis Nodes
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Master
KeepAliveD...
23
Fail Over – Redis Nodes
STASH NOVA
Proxy
API
Sentinels
Host 0
Host 1
Host 2
Host 3
Redis
Master
Redis
Master
KeepAliveD...
Opportunities
•  Failover is easy
•  Easy to introduce changes
•  Reusable architecture
24
25
Redis Resize – Vertical Scaling
STASH
Sentinels
Redis
Master
Redis
Slave
HA
Proxy
HA
Proxy
Customer :6379
26
Redis Resize – Vertical Scaling
STASH
Sentinels
Redis
Master
Redis
Slave
HA
Proxy
HA
Proxy
Customer :6379
New
Redis
Mas...
27
Redis Resize – Vertical Scaling
STASH
Sentinels
Redis
Master
Redis
Slave
HA
Proxy
HA
Proxy
Customer :6379
New
Redis
Mas...
28
Redis Resize – Vertical Scaling
STASH
Sentinels
HA
Proxy
HA
Proxy
Customer :6379
New
Redis
Master
New
Redis
Slave
Opportunities
•  Failover is easy
•  Easy to introduce changes
•  Reusable architecture
29
Highly Available MySQL
30
31
Architecture - Challenges
Challenges
•  Global sentinels
•  Redis drivers
32
Global Sentinels
33
34
Sentinels
Redis Drivers/Clients
35
36
Architecture - Case Study
Tale of ‘Resize Why you no work?’
37
38
Redis Resize – Vertical Scaling
STASH
Sentinels
Redis
Master
Redis
Slave
HA
Proxy
HA
Proxy
Customer :6379
New
Redis
Mas...
Outline
• Architecture
• Security and Isolation
• Know your environment
39
40
Security and Isolation
ACLs
41
42
VIP
LB1 (Master)
HAProxy
ACLS
LB2
(Backup)
HAProxy
ACLs
keepalived
Redis
Master
Redis
Slave
Access HA Redis:
VIP: read/...
SSL
43
44
VIP
LB1 (Master)
HAProxy
ACLs
SSL certs
LB2
(Backup)
HAProxy
ACLs
SSL certs
keepalived
Redis
Master
Redis
Slave
Access ...
Containers
45
46
VIP
LB1 (Master)
HAProxy
LB2
(Backup)
HAProxy
keepalived
Redis
Master
Redis
Slave
Access HA Redis:
VIP: read/write
port...
47
Security and Isolation – Case Study
48
VIP
LB1 (Master)
HAProxy
LB2
(Backup)
HAProxy
keepalived
Redis
Master
Redis
Slave
Access HA Redis:
VIP: read/write
port...
Outline
• Architecture
• Security and Isolation
• Know thy environment
49
50
Monitoring, Monitoring, Monitoring
•  What do you monitor?
•  How often is it monitored?
•  Who does the monitoring?
•  Wh...
Monitoring, Monitoring, Monitoring
•  What do you monitor?
–  Front door check
–  Connection limit
–  Memory usage
–  Fail...
Monitoring, Monitoring, Monitoring
•  What do you monitor?
•  How often is it monitored?
–  Front door check
–  Memory usa...
Monitoring, Monitoring, Monitoring
•  What do you monitor?
•  How often is it monitored?
•  Who does the monitoring?
•  Wh...
55
56
Monitoring, Monitoring, Monitoring
•  What do you monitoring?
•  How often is it monitored?
•  Who does the monitoring?
• ...
58
Monitoring, Monitoring, Monitoring
•  What do you monitoring?
•  How often is it monitored?
•  Who does the monitoring?
• ...
60
Know Thy Environment - Challenges
61
62
Know Thy Environment – Case Studies
Tale of the customer saved by Zendesk
63
Outline
• Architecture
• Security and Isolation
• Know thy environment
64
65
66
Upcoming SlideShare
Loading in …5
×

Redis in a Multi Tenant Environment–High Availability, Monitoring & Much More!

3,845 views

Published on

Running any
application in a multi-tenant environment poses its challenges. This talk is focused around how we at Rackspace run Redis
in a multi-tenant environment, ensuring security, performance, fault tolerance and high availability. This talk will cover: an
architecture deep dive of multi tenant Redis on the cloud, management of sentinels, monitoring and operations of a large
scale Redis deployment,introducing new Redis versions,scaling, security, some lessons learnt. The target audience for this
talk is anyone who is interested in the deployment/operational aspect of running Redis. This is relevant not only for those
who want to run Redis themselves, but also interested in how a Redis provider might be doing it for them.

Published in: Technology
  • I think you need a perfect and 100% unique academic essays papers have a look once this site i hope you will get valuable papers, HelpWriting.net
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❤❤❤ http://bit.ly/2u6xbL5 ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❤❤❤ http://bit.ly/2u6xbL5 ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Redis in a Multi Tenant Environment–High Availability, Monitoring & Much More!

  1. 1. REDIS IN A MULTI-TENANT ENVIRONMENT Iccha Sethi, Software Developer Twitter: @IcchaSethi
  2. 2. 2 IcchaSethi.__dict__ { “Name”: “Iccha Sethi”, “Occupation”: “Software Developer”, “Team”: “Redis”, “Company”: “Object Rocket, Rackspace”, “Fun Fact”: “loading….” }
  3. 3. Why does High Availability matter for Redis? 3
  4. 4. Why does High Availability matter for Redis? 4
  5. 5. Outline • Architecture • Security and Isolation • Know thy environment 5
  6. 6. Outline • Architecture • Security and Isolation • Know thy environment 6
  7. 7. 7 Architecture
  8. 8. Outline 8 Redis Master Access HA Redis: Dns: read/write port 6379 password
  9. 9. Outline 9 Redis Master Redis Slave Access HA Redis: DNS: read/write port 6379 password
  10. 10. Outline 10 VIP LB1 (Master) HAProxy Redis Master Redis Slave Access HA Redis: DNS: read/write port 6379 password
  11. 11. Outline 11 VIP LB1 (Master) HAProxy LB2 (Backup) HAProxy keepalived Redis Master Redis Slave Access HA Redis: DNS: read/write port 6379 password
  12. 12. 12 Sentinels
  13. 13. 13 Highly Available REDIS STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Slave KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379 HA Create
  14. 14. 14 Architecture - Opportunities
  15. 15. Opportunities •  Failover is easy •  Easy to introduce changes •  Reusable architecture 15
  16. 16. Opportunities •  Failover is easy •  Easy to introduce changes •  Reusable architecture 16
  17. 17. 17 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Slave KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
  18. 18. 18 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Slave KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
  19. 19. 19 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
  20. 20. 20 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
  21. 21. 21 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
  22. 22. 22 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
  23. 23. 23 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
  24. 24. Opportunities •  Failover is easy •  Easy to introduce changes •  Reusable architecture 24
  25. 25. 25 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379
  26. 26. 26 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
  27. 27. 27 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
  28. 28. 28 Redis Resize – Vertical Scaling STASH Sentinels HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
  29. 29. Opportunities •  Failover is easy •  Easy to introduce changes •  Reusable architecture 29
  30. 30. Highly Available MySQL 30
  31. 31. 31 Architecture - Challenges
  32. 32. Challenges •  Global sentinels •  Redis drivers 32
  33. 33. Global Sentinels 33
  34. 34. 34 Sentinels
  35. 35. Redis Drivers/Clients 35
  36. 36. 36 Architecture - Case Study
  37. 37. Tale of ‘Resize Why you no work?’ 37
  38. 38. 38 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
  39. 39. Outline • Architecture • Security and Isolation • Know your environment 39
  40. 40. 40 Security and Isolation
  41. 41. ACLs 41
  42. 42. 42 VIP LB1 (Master) HAProxy ACLS LB2 (Backup) HAProxy ACLs keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 ACLs
  43. 43. SSL 43
  44. 44. 44 VIP LB1 (Master) HAProxy ACLs SSL certs LB2 (Backup) HAProxy ACLs SSL certs keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 SSL port 6380 SSL
  45. 45. Containers 45
  46. 46. 46 VIP LB1 (Master) HAProxy LB2 (Backup) HAProxy keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 Containers
  47. 47. 47 Security and Isolation – Case Study
  48. 48. 48 VIP LB1 (Master) HAProxy LB2 (Backup) HAProxy keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 Tale of the Customer with 120K connections
  49. 49. Outline • Architecture • Security and Isolation • Know thy environment 49
  50. 50. 50
  51. 51. Monitoring, Monitoring, Monitoring •  What do you monitor? •  How often is it monitored? •  Who does the monitoring? •  What action is taken? 51
  52. 52. Monitoring, Monitoring, Monitoring •  What do you monitor? –  Front door check –  Connection limit –  Memory usage –  Failover –  Failed failover –  Redis process down –  No valid slaves •  How often is it monitored? •  Who does the monitoring? •  What action is taken? 52
  53. 53. Monitoring, Monitoring, Monitoring •  What do you monitor? •  How often is it monitored? –  Front door check –  Memory usage –  Failover –  Failed failover –  Redis process down –  No valid slaves •  Who does the monitoring? •  What action is taken? 53
  54. 54. Monitoring, Monitoring, Monitoring •  What do you monitor? •  How often is it monitored? •  Who does the monitoring? •  What action is taken? 54
  55. 55. 55
  56. 56. 56
  57. 57. Monitoring, Monitoring, Monitoring •  What do you monitoring? •  How often is it monitored? •  Who does the monitoring? •  What action is taken? 57
  58. 58. 58
  59. 59. Monitoring, Monitoring, Monitoring •  What do you monitoring? •  How often is it monitored? •  Who does the monitoring? •  What action is taken? –  Front door check –  Memory usage –  Failover –  Failed failover –  Redis process down –  No valid slaves 59
  60. 60. 60 Know Thy Environment - Challenges
  61. 61. 61
  62. 62. 62 Know Thy Environment – Case Studies
  63. 63. Tale of the customer saved by Zendesk 63
  64. 64. Outline • Architecture • Security and Isolation • Know thy environment 64
  65. 65. 65
  66. 66. 66

×