Redis       @http://weibo.com/tangfl     http://fulin.org
• Redis•         Redis  •  ••
Redis• in memory (database?)• data can dump to disk• many useful data structure• FAST both read and write• we start use fr...
=feed+   +
=feed+   +• feed • •
=feed+   +
=feed+   +•    •    •
=feed+   +
=feed+   +•    •    •       @    •            @    •
=feed+   +
=feed+   +•    •    •       @    •            @    •
=feed+   +
=feed+   +•    •    •       @    •            @    •
=feed+   +
=feed+   +•    •    •       @    •            @    •
=feed+   +
=feed+   +•feed • mysql • mc
=feed+   +•
=feed+                                   +• mysql: relation.following    •   fromuid, touid, addtime•   addtime desc      ...
=feed+                                   +• mysql: relation.following relation.follower    •   fromuid, touid, addtime•   ...
=feed+   +•    •        •    •        •
=feed+        +•   ta•                  ta•             ta•
=feed+     +•    •     c/java    •    •    •
=feed+          +• redis• hash : • key : user id • fields : friends ids • value : add time
=feed+                            +• redis• hash : • hset fromuid.following touid addtime • hset touid.follower fromuid ad...
=feed+             +• hash-max-zip-size •   64 -> 256   1/3 •   cpu• hgetAll cost too much cpu • add mc• high delay
=feed+   +
=feed+                        +• redis • cache ? waste too much mem • storage ?   • rdb may lost data   • aof r/w too slow...
=feed+   +
=feed+                  +•    • mysql binlog >> queue >> Java Processor      >> redis    • mysql binlog >> trigger >> redis
=feed+             +•    • redis@weibo for now:    • TB    • growing fast
=feed+                  +•    •@               mysql modified + innodb    • following list of a user in one column    • sti...
=feed+   +•
=feed+   +•    •        •        •@    •        •
=feed+   +
=feed+    +•    •        •         @    •        •        •    js
=feed+           +• mc + queue + mysql•         mysql• mc mysql•                  mc
=feed+   +•    •    •    •    •
=feed+             +• redis • k-v , 100 byte per k-v   •   mc   • • hash , hget pipeline slow
=feed+                +• redis • rdb ? may lost data • aof ? grow too fast (4G/day) • bgsave/bgrewriteaof influence parent
=feed+   +
=feed+         +• redis rolling  •  •  •  •               key
=feed+                 +•    • rediscounter @    • array , not linked list    • malloc all mem when start    • hash key to...
=feed+                   +•    • rediscounter + innodb    • auto roll cold data to disk
=feed+                                     +•    •   @TimYang    •   @XiaoJunHong    •    •   redis (delay) - mc (throughp...
•    • cache    • storage
•    •    •    •   +    •
•    •    •   /    •    •
•    •    • rdb or aof
•    •    • rehash
• HA / Cluster •       Redis • Jedis
• CPU • Redis • hset with big hash-max-zip-size • hgetAll •         mc
•    •    •    •    •
Thanks       @http://weibo.com/tangfl     http://fulin.org
Q &A   PS. We are hiring !contact me via @
Upcoming SlideShare
Loading in …5
×

新浪微博开放平台中的 Redis 实践

6,431 views

Published on

2011年10月,Qcon 杭州,开放平台话题,新浪微博开放中的 Redis 实践,唐福林

Published in: Technology, Business
0 Comments
16 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,431
On SlideShare
0
From Embeds
0
Number of Embeds
3,799
Actions
Shares
0
Downloads
141
Comments
0
Likes
16
Embeds 0
No embeds

No notes for slide

新浪微博开放平台中的 Redis 实践

  1. 1. Redis @http://weibo.com/tangfl http://fulin.org
  2. 2. • Redis• Redis • ••
  3. 3. Redis• in memory (database?)• data can dump to disk• many useful data structure• FAST both read and write• we start use from 2.0, now 2.4
  4. 4. =feed+ +
  5. 5. =feed+ +• feed • •
  6. 6. =feed+ +
  7. 7. =feed+ +• • •
  8. 8. =feed+ +
  9. 9. =feed+ +• • • @ • @ •
  10. 10. =feed+ +
  11. 11. =feed+ +• • • @ • @ •
  12. 12. =feed+ +
  13. 13. =feed+ +• • • @ • @ •
  14. 14. =feed+ +
  15. 15. =feed+ +• • • @ • @ •
  16. 16. =feed+ +
  17. 17. =feed+ +•feed • mysql • mc
  18. 18. =feed+ +•
  19. 19. =feed+ +• mysql: relation.following • fromuid, touid, addtime• addtime desc select * from following where fromuid=? order by• addtime desc select * from following where touid=? order by• fromuid, touid
  20. 20. =feed+ +• mysql: relation.following relation.follower • fromuid, touid, addtime• addtime desc select * from following where fromuid=? order by• desc select * from follower where touid=? order by addtime•
  21. 21. =feed+ +• • • • •
  22. 22. =feed+ +• ta• ta• ta•
  23. 23. =feed+ +• • c/java • • •
  24. 24. =feed+ +• redis• hash : • key : user id • fields : friends ids • value : add time
  25. 25. =feed+ +• redis• hash : • hset fromuid.following touid addtime • hset touid.follower fromuid addtime • hgetAll fromuid. following • hgetAll touid.follower ? • 11,704,598 @Wed Sep 7 21:46:33 CST 2011
  26. 26. =feed+ +• hash-max-zip-size • 64 -> 256 1/3 • cpu• hgetAll cost too much cpu • add mc• high delay
  27. 27. =feed+ +
  28. 28. =feed+ +• redis • cache ? waste too much mem • storage ? • rdb may lost data • aof r/w too slow, recover too slow • all data in mem, waste money • HA : master slave ? NO WAY • memory fragment
  29. 29. =feed+ +
  30. 30. =feed+ +• • mysql binlog >> queue >> Java Processor >> redis • mysql binlog >> trigger >> redis
  31. 31. =feed+ +• • redis@weibo for now: • TB • growing fast
  32. 32. =feed+ +• •@ mysql modified + innodb • following list of a user in one column • still under dev
  33. 33. =feed+ +•
  34. 34. =feed+ +• • • •@ • •
  35. 35. =feed+ +
  36. 36. =feed+ +• • • @ • • • js
  37. 37. =feed+ +• mc + queue + mysql• mysql• mc mysql• mc
  38. 38. =feed+ +• • • • •
  39. 39. =feed+ +• redis • k-v , 100 byte per k-v • mc • • hash , hget pipeline slow
  40. 40. =feed+ +• redis • rdb ? may lost data • aof ? grow too fast (4G/day) • bgsave/bgrewriteaof influence parent
  41. 41. =feed+ +
  42. 42. =feed+ +• redis rolling • • • • key
  43. 43. =feed+ +• • rediscounter @ • array , not linked list • malloc all mem when start • hash key to position • write disk: asyn & slow down • add position to aof file
  44. 44. =feed+ +• • rediscounter + innodb • auto roll cold data to disk
  45. 45. =feed+ +• • @TimYang • @XiaoJunHong • • redis (delay) - mc (throughput) - java hash map
  46. 46. • • cache • storage
  47. 47. • • • • + •
  48. 48. • • • / • •
  49. 49. • • • rdb or aof
  50. 50. • • • rehash
  51. 51. • HA / Cluster • Redis • Jedis
  52. 52. • CPU • Redis • hset with big hash-max-zip-size • hgetAll • mc
  53. 53. • • • • •
  54. 54. Thanks @http://weibo.com/tangfl http://fulin.org
  55. 55. Q &A PS. We are hiring !contact me via @

×