“Redis ismore than justa key-value store,it’s a lifestyle.            –– Mathias Meyer
Fundamental data structuresfor a fundamental API    a data structure server      lists                     sets    sorted ...
user1234567890field0:   abc…field1:   def…field2:   ghi…field3:   jkl…field4:   mno…field5:   pqr…field6:   stu…field7:   ...
user1234567890field0: abc…                                            hashfield1: def…field2: ghi…field3: jkl…            ...
Fundamental data structuresfor a fundamental API    a data structure server      lists                     sets    sorted ...
factory settings            hand-tuned            0.5M initial size   2.54MB                   1.62MB             hash-max...
The Harry Potter ApproachWWVDWhat Would Voldemort Do101 @Override102 public int scan(…) {104 logger.warn(⁞     "Voldemort ...
sorted sets
user1   user2   user3   user5   user8 1       2       3        5      8…
user1   user2   user3   user5   user8        not that easy
record name can be anything  thus: position = hash(name)user1   user2   user3   user5   user8        not that easy
sorted         sets50% performance hit
workload A    throughput                 target
workload E    throughput                 target
workload F    throughput                 target
“I think the… problem is…believing thatthere can be“one true datastore”.Different technologiesexcel at different things.  ...
“I think the… problem is…believing thatthere can be“one true datastore”.Different technologiesexcel at different things.  ...
Memory storage is #1                                                           Redis key spacehttp://spotfireblog.tibco.co...
network bandwidth contentionmemory
200 ops/sec single-threaded
user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu…                        ...
user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu…                        ...
user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu…                        ...
user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu…                        ...
user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu…                        ...
user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu…                        ...
32-bit      64-bit4 GB         8 TB          large pointers
redis-cli INFO
Memory storage is #1               meh, persistence                                                          memory       ...
Memory storage is #1               meh, persistence         load phase fails randomly                        memory       ...
there are no silver bullets
Redis on AWS
Redis on AWS
Upcoming SlideShare
Loading in …5
×

Redis on AWS

6,866 views

Published on

Presentation held in the seminar on "NotOnlySQL databases." Builds upon slides in the former presentation "Redis — memcached on steroids." Features challenges when deploying Redis on Amazon Web Services within the context of the Yahoo! Cloud Serving Benchmark.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,866
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
25
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Redis on AWS

  1. 1. “Redis ismore than justa key-value store,it’s a lifestyle. –– Mathias Meyer
  2. 2. Fundamental data structuresfor a fundamental API a data structure server lists sets sorted hash sets tables
  3. 3. user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu…field7: vwx…field8: yzä…field9: öüß…
  4. 4. user1234567890field0: abc… hashfield1: def…field2: ghi…field3: jkl… RedisClient.java tablesfield4: mno…field5: pqr…field6: stu…field7: vwx…field8: yzä…field9: öüß… insert HMSET read HMGET update HMSET delete DEL
  5. 5. Fundamental data structuresfor a fundamental API a data structure server lists sets sorted hash sets tables
  6. 6. factory settings hand-tuned 0.5M initial size 2.54MB 1.62MB hash-max-zipmap-value 2048
  7. 7. The Harry Potter ApproachWWVDWhat Would Voldemort Do101 @Override102 public int scan(…) {104 logger.warn(⁞ "Voldemort does not support Scan semantics");105 return OK;106 } db/voldemort/src/com/yahoo/ycsb/db/VoldemortClient.java
  8. 8. sorted sets
  9. 9. user1 user2 user3 user5 user8 1 2 3 5 8…
  10. 10. user1 user2 user3 user5 user8 not that easy
  11. 11. record name can be anything thus: position = hash(name)user1 user2 user3 user5 user8 not that easy
  12. 12. sorted sets50% performance hit
  13. 13. workload A throughput target
  14. 14. workload E throughput target
  15. 15. workload F throughput target
  16. 16. “I think the… problem is…believing thatthere can be“one true datastore”.Different technologiesexcel at different things. –– Weixi Yen
  17. 17. “I think the… problem is…believing thatthere can be“one true datastore”.Different technologiesexcel at different things. –– Weixi Yen
  18. 18. Memory storage is #1 Redis key spacehttp://spotfireblog.tibco.com/wp-content/uploads/in-memory-analytics.jpg
  19. 19. network bandwidth contentionmemory
  20. 20. 200 ops/sec single-threaded
  21. 21. user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu… = 1 KBfield7: vwx…field8: yzä…field9: öüß…
  22. 22. user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu… = 1 KBfactory page size := 32 bytesfield7: vwx…field8: yzä…field9: öüß…
  23. 23. user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu… = 1 KBfactory page size := 32 bytesfield7: vwx…field8: yzä…field9: öüß…values must be swapped in their entirety, even for complex types
  24. 24. user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu… = 1 KBfactory page size := 32 bytesfield7: vwx…field8: yzä…field9: öüß…values must be swapped in their entirety, even for complex types = 32 pages 
  25. 25. user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu… = 1 KBfactory page size := 32 bytesfield7: vwx…field8: yzä…field9: öüß…values must be swapped in their entirety, even for complex types = 32 pages  vm-page-size et al
  26. 26. user1234567890field0: abc…field1: def…field2: ghi…field3: jkl…field4: mno…field5: pqr…field6: stu… = 1 KBfield7: vwx…field8: yzä… 50% performance hitfield9: öüß… = 32 pages 
  27. 27. 32-bit 64-bit4 GB 8 TB large pointers
  28. 28. redis-cli INFO
  29. 29. Memory storage is #1 meh, persistence memory overheadhttp://www.flickr.com/photos/generationbass/4827013488/
  30. 30. Memory storage is #1 meh, persistence load phase fails randomly memory overheadhttp://www.flickr.com/photos/generationbass/4827013488/
  31. 31. there are no silver bullets

×