Redis in a Multi Tenant Environment–High Availability, Monitoring & Much More!
•
10 likes•4,542 views
This document discusses best practices for running Redis in a multi-tenant environment. It covers architectural considerations like high availability, security and isolation techniques using ACLs and SSL, and the importance of monitoring and understanding your environment. The key opportunities are that failover is easy to implement, changes can be introduced smoothly, and the architecture is reusable. Challenges include managing global sentinels and Redis drivers. Case studies demonstrate issues like customer spikes causing problems and the importance of monitoring.
More Related Content
What's hot
What's hot (20)
Similar to Redis in a Multi Tenant Environment–High Availability, Monitoring & Much More!
Similar to Redis in a Multi Tenant Environment–High Availability, Monitoring & Much More! (20)
More from Redis Labs
More from Redis Labs (20)
Recently uploaded
Recently uploaded (20)
Redis in a Multi Tenant Environment–High Availability, Monitoring & Much More!
- 1. REDIS IN A MULTI-TENANT ENVIRONMENT Iccha Sethi, Software Developer Twitter: @IcchaSethi
- 2. 2 IcchaSethi.__dict__ { “Name”: “Iccha Sethi”, “Occupation”: “Software Developer”, “Team”: “Redis”, “Company”: “Object Rocket, Rackspace”, “Fun Fact”: “loading….” }
- 3. Why does High Availability matter for Redis? 3
- 4. Why does High Availability matter for Redis? 4
- 5. Outline • Architecture • Security and Isolation • Know thy environment 5
- 6. Outline • Architecture • Security and Isolation • Know thy environment 6
- 8. Outline 8 Redis Master Access HA Redis: Dns: read/write port 6379 password
- 9. Outline 9 Redis Master Redis Slave Access HA Redis: DNS: read/write port 6379 password
- 10. Outline 10 VIP LB1 (Master) HAProxy Redis Master Redis Slave Access HA Redis: DNS: read/write port 6379 password
- 11. Outline 11 VIP LB1 (Master) HAProxy LB2 (Backup) HAProxy keepalived Redis Master Redis Slave Access HA Redis: DNS: read/write port 6379 password
- 12. 12 Sentinels
- 13. 13 Highly Available REDIS STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Slave KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379 HA Create
- 15. Opportunities • Failover is easy • Easy to introduce changes • Reusable architecture 15
- 16. Opportunities • Failover is easy • Easy to introduce changes • Reusable architecture 16
- 17. 17 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Slave KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
- 18. 18 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Slave KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
- 19. 19 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
- 20. 20 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
- 21. 21 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
- 22. 22 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
- 23. 23 Fail Over – Redis Nodes STASH NOVA Proxy API Sentinels Host 0 Host 1 Host 2 Host 3 Redis Master Redis Master KeepAliveD HA Proxy KeepAliveD HA Proxy Proxy Agent Proxy Agent Agent Agent Customer :6379
- 24. Opportunities • Failover is easy • Easy to introduce changes • Reusable architecture 24
- 25. 25 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379
- 26. 26 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
- 27. 27 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
- 28. 28 Redis Resize – Vertical Scaling STASH Sentinels HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
- 29. Opportunities • Failover is easy • Easy to introduce changes • Reusable architecture 29
- 32. Challenges • Global sentinels • Redis drivers 32
- 34. 34 Sentinels
- 36. 36 Architecture - Case Study
- 37. Tale of ‘Resize Why you no work?’ 37
- 38. 38 Redis Resize – Vertical Scaling STASH Sentinels Redis Master Redis Slave HA Proxy HA Proxy Customer :6379 New Redis Master New Redis Slave
- 39. Outline • Architecture • Security and Isolation • Know your environment 39
- 41. ACLs 41
- 42. 42 VIP LB1 (Master) HAProxy ACLS LB2 (Backup) HAProxy ACLs keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 ACLs
- 43. SSL 43
- 44. 44 VIP LB1 (Master) HAProxy ACLs SSL certs LB2 (Backup) HAProxy ACLs SSL certs keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 SSL port 6380 SSL
- 45. Containers 45
- 46. 46 VIP LB1 (Master) HAProxy LB2 (Backup) HAProxy keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 Containers
- 47. 47 Security and Isolation – Case Study
- 48. 48 VIP LB1 (Master) HAProxy LB2 (Backup) HAProxy keepalived Redis Master Redis Slave Access HA Redis: VIP: read/write port 6379 Tale of the Customer with 120K connections
- 49. Outline • Architecture • Security and Isolation • Know thy environment 49
- 50. 50
- 51. Monitoring, Monitoring, Monitoring • What do you monitor? • How often is it monitored? • Who does the monitoring? • What action is taken? 51
- 52. Monitoring, Monitoring, Monitoring • What do you monitor? – Front door check – Connection limit – Memory usage – Failover – Failed failover – Redis process down – No valid slaves • How often is it monitored? • Who does the monitoring? • What action is taken? 52
- 53. Monitoring, Monitoring, Monitoring • What do you monitor? • How often is it monitored? – Front door check – Memory usage – Failover – Failed failover – Redis process down – No valid slaves • Who does the monitoring? • What action is taken? 53
- 54. Monitoring, Monitoring, Monitoring • What do you monitor? • How often is it monitored? • Who does the monitoring? • What action is taken? 54
- 55. 55
- 56. 56
- 57. Monitoring, Monitoring, Monitoring • What do you monitoring? • How often is it monitored? • Who does the monitoring? • What action is taken? 57
- 58. 58
- 59. Monitoring, Monitoring, Monitoring • What do you monitoring? • How often is it monitored? • Who does the monitoring? • What action is taken? – Front door check – Memory usage – Failover – Failed failover – Redis process down – No valid slaves 59
- 60. 60 Know Thy Environment - Challenges
- 61. 61
- 62. 62 Know Thy Environment – Case Studies
- 63. Tale of the customer saved by Zendesk 63
- 64. Outline • Architecture • Security and Isolation • Know thy environment 64
- 65. 65
- 66. 66