HIgh Performance Redis- Tague Griffith, GoPro

344 views

Published on

High Performance Redis looks at a wide range of techniques - from programming to system tuning - to deploy and maintain an extremely high performing Redis cluster. From the operational
perspective, the talk lays out multiple techniques for clustering (sharding) Redis systems and examines how the different
approaches impact performance time. The talk further examines system settings (Linux network parameters, Redis
system) and how they impact performance (both good and bad). Finally, for the developer, we look at how different ways of structuring data actually demonstrate different performance characteristics

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
344
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

HIgh Performance Redis- Tague Griffith, GoPro

  1. 1. High Performance Redis Tague Griffith
  2. 2. Development qProgramming qData restructuring qTransactions High Performance Redis Holistic understanding of some techniques and tricks for building out high performance Redis systems. Agenda Technology qRedis 2.x qTwemproxy qLinux Operations qClustering qFailover qOS Tuning qRedis Tuning
  3. 3. Introductions
  4. 4. • Software Architect with GoPro • 10+ years building distributed systems • 4+ years building with Redis • Built multiple high performing Redis apps • Previously: Apple, Netscape, Flickr Who Am I
  5. 5. Who Are You? Loki The Wolfdog
  6. 6. What is High Performance
  7. 7. • Large number of clients • High transaction rate • Low response time • Highly Available System Features
  8. 8. Main Techniques ü Large number of clients ü High transaction rate ü Low response time ü High Availability ü Low Response time ü High transaction rate ü Low response time Clustering and Failover System Tuning Programming Techniques
  9. 9. Clustering
  10. 10. • Large number of clients • High transaction rate • Low response time • Pre-requisite to high availability Features
  11. 11. Basic Sharding Client Client Client Client Client Shard-0 Shard-1 Shard-n • Read/Writes same node • Uniform workloads • Key based partitioning
  12. 12. • Needs a uniform work load • Great for single key operations • Traversals are expensive • Node loss – roughly 1/N complete outage • Hot keys are an issue Trade-offs
  13. 13. Read/Write Replicas Client Client Client Client Client Write Master Read Secondary Read Secondary • Read/Writes split • High Read workloads • Different than Failover Replicas
  14. 14. • Works well for high read workload • Replication delay • Lose read your writes consistency • Node loss - depends on the node • Easy to scale up hot keys Trade-offs
  15. 15. Duplicate Nodes Client Client Client ClientWrite Client Node-0 Node-1 Client Client Client ClientRead Client Read Read 1st Result
  16. 16. • High performance reads • Significant client coding • Consistency goes out the window • Node loss – depends on your approach • Double the cost Trade-offs
  17. 17. • Clustering techniques can be combined • Sharding + R/W Replicas • Sharding + Multi-Node • Multi-node is a technique of last resort Hybrids
  18. 18. • Scale horizontally not vertically • Use multiple Redis instances per host • R/W replicas or dedicated shards for hot keys • Automate deployment and configuration Scaling Up
  19. 19. Happily Ever After ü Large Number of Clients ü High Transaction Rate q Highly Available q Low Response Time Let’s Party
  20. 20. Failover
  21. 21. • Twemproxy • Sentinel Out of the Box Failover Systems
  22. 22. Happily Ever After ü Large Number of Clients ü High Transaction Rate ü Highly Available q Low Response Time Let’s Party
  23. 23. System Tuning
  24. 24. • Network tuning • Persistence settings • Small Aggregate Data settings Tuning
  25. 25. • net.ipv4.ip_local_port_range • net.ipv4.tcp_tw_recycle • net.ipv4.tcp_tw_reuse • net.core.somaxconn Network Tuning - Linux
  26. 26. • tcp-backlog • tcp-keepalive Network Tuning - Redis
  27. 27. • hash-max-zipmap-entries • hash-max-zipmap-value • list-max-ziplist-entries • list-max-ziplist-value • zset-max-ziplist-entries • zset-max-ziplist-value • set-max-intset-entries Small Aggregate Data Switches
  28. 28. • RDB • Fork and Save • AOF • On “client” thread • Fsync policy Persistence
  29. 29. Happily Ever After ü Large Number of Clients ü High Transaction Rate ü Highly Available q Low Response Time Let’s Party
  30. 30. Programming Techniques
  31. 31. • Connection Management • Transaction Structure • Data Layout Key Areas to Consider
  32. 32. Persistent Connections • Setup is expensive • Reduced latency • Server tuning maybe required Connection Management Operation Pipelining • Substantial speedup (ops/s) • Failure harder to manage • Client support required Connection Pooling • Form of persistent connections • Reduced latency • Client support required
  33. 33. The structure of your Redis transactions can greatly effect the performance of your app. Transaction Structure Smaller • Lower latency • Improved responsiveness • Reduced throughput • Negative impact onAOF Larger • Higher latency • Reduced responsiveness • Increased throughput • Less impact on AOF
  34. 34. Main Techniques • Smaller Structures faster • Pops off of long lists slow • Internally “shard” data if possible • replace strings with denser coding (bit vector, map, etc) • Moderate sized structures • Reduce size of modifications • Drop keys v. mutating • Reduce impact on AOF Structured Types Memory Usage Isolate Modifications
  35. 35. Happily Ever After ü Large Number of Clients ü High Transaction Rate ü Highly Available ü Low Response Time Let’s Party
  36. 36. Hey, What about My System?
  37. 37. • All environments are different • Unexpected latency will make you cry • For performance, scale horizontally • Monitoring and Tools are Essential • Connections are expensive • Redundancy and Failover needs to be automatic • AOF is a fair weather friend Take Aways
  38. 38. Questions?
  39. 39. Thanks!

×