新浪微博开放平台Redis实战

41,135 views

Published on

新浪微博开放平台redis实战

Published in: Technology, Business
3 Comments
165 Likes
Statistics
Notes
No Downloads
Views
Total views
41,135
On SlideShare
0
From Embeds
0
Number of Embeds
23,430
Actions
Shares
0
Downloads
0
Comments
3
Likes
165
Embeds 0
No embeds

No notes for slide

新浪微博开放平台Redis实战

  1. 1. Redis @http://weibo.com/tangfl http://fulin.org
  2. 2. • Redis• Redis • • redis • redis•
  3. 3. Redis• in memory (database?)• data can dump to disk• many useful data structure• FAST both read and write• we start use from 2.0, now 2.4
  4. 4. =feed+ +
  5. 5. =feed+ +• feed • •
  6. 6. =feed+ +
  7. 7. =feed+ +• • •
  8. 8. =feed+ +
  9. 9. =feed+ +• • • @ • @ •
  10. 10. =feed+ +
  11. 11. =feed+ +• • • @ • @ •
  12. 12. =feed+ +
  13. 13. =feed+ +• • • @ • @ •
  14. 14. =feed+ +
  15. 15. =feed+ +• • • @ • @ •
  16. 16. =feed+ +
  17. 17. =feed+ +•feed • mysql • mc
  18. 18. =feed+ +•
  19. 19. =feed+ +• mysql: relation.following • fromuid, touid, addtime• addtime desc select * from following where fromuid=? order by• addtime desc select * from following where touid=? order by• fromuid, touid
  20. 20. =feed+ +• mysql: relation.following relation.follower • fromuid, touid, addtime• addtime desc select * from following where fromuid=? order by• desc select * from follower where touid=? order by addtime•
  21. 21. =feed+ +• • • • •
  22. 22. =feed+ +• ta• ta• ta•
  23. 23. =feed+ +• • c/java • • •
  24. 24. =feed+ +• redis• hash : • key : user id • fields : friends ids • value : add time
  25. 25. =feed+ +• redis• hash : • hset fromuid.following touid addtime • hset touid.follower fromuid addtime • hgetAll fromuid. following • hgetAll touid.follower ? • 11,704,598 @Wed Sep 7 21:46:33 CST 2011
  26. 26. =feed+ +• hash: get one or get all• hash-max-zip-size • 64 -> 256 1/3 • cpu• hgetAll cost too much cpu • add mc• high delay
  27. 27. =feed+ +
  28. 28. =feed+ +• • cache ? waste too much mem • storage ? • rdb may lost data • aof r/w too slow, recover too slow • all data in mem, waste money • HA : master slave ? NO WAY • memory fragment
  29. 29. =feed+ +
  30. 30. =feed+ +• • mysql binlog >> queue >> Java Processor >> redis • mysql binlog >> trigger >> redis
  31. 31. =feed+ +• • redis@weibo for now: • TB • growing fast
  32. 32. =feed+ +• •@ mysql modified + innodb • following list of a user in one column • still under dev
  33. 33. =feed+ +•
  34. 34. =feed+ +• • • •@ • •
  35. 35. =feed+ +
  36. 36. =feed+ +• • • @ • • • js
  37. 37. =feed+ +• mc + queue + mysql• mysql• mc mysql• mc
  38. 38. =feed+ +• • • • •
  39. 39. =feed+ +• redis• • k-v , 100 byte per k-v • mc • • hash , hget pipeline slow
  40. 40. =feed+ +• • rdb ? may lost data • aof ? grow too fast (4G/day) • bgsave/bgrewriteaof influence parent
  41. 41. =feed+ +
  42. 42. =feed+ +• redis rolling • • • • key
  43. 43. =feed+ +• • rediscounter @ • array , not linked list • malloc all mem when start • hash key to position • write disk: asyn & slow down • add position to aof file
  44. 44. =feed+ +• • rediscounter + innodb • auto roll cold data to disk
  45. 45. =feed+ +• • @TimYang • @XiaoJunHong • • redis (delay) - mc (throughput) - java hash map
  46. 46. • • cache • storage
  47. 47. • • • • + •
  48. 48. • • • / • •
  49. 49. • • • rdb or aof • •
  50. 50. • • • rehash
  51. 51. • HA / Cluster • Redis • Jedis • • •
  52. 52. • CPU • Redis • hset with big hash-max-zip-size • hgetAll • mc
  53. 53. • • • • •
  54. 54. Thanks PS. We are hiring !contact me via @
  55. 55. Q &A @http://weibo.com/tangfl http://fulin.org

×