新浪微博开放平台Redis实战

Redis @http://weibo.com/tangﬂ http://fulin.org

• Redis• Redis • • redis • redis•

Redis• in memory (database?)• data can dump to disk• many useful data structure• FAST both read and write• we start use from 2.0, now 2.4

=feed+ +

=feed+ +• feed • •

=feed+ +

=feed+ +• • •

=feed+ +

=feed+ +• • • @ • @ •

=feed+ +

=feed+ +• • • @ • @ •

=feed+ +

=feed+ +• • • @ • @ •

=feed+ +

=feed+ +• • • @ • @ •

=feed+ +

=feed+ +•feed • mysql • mc

=feed+ +•

=feed+ +• mysql: relation.following • fromuid, touid, addtime• addtime desc select * from following where fromuid=? order by• addtime desc select * from following where touid=? order by• fromuid, touid

=feed+ +• mysql: relation.following relation.follower • fromuid, touid, addtime• addtime desc select * from following where fromuid=? order by• desc select * from follower where touid=? order by addtime•

=feed+ +• • • • •

=feed+ +• ta• ta• ta•

=feed+ +• • c/java • • •

=feed+ +• redis• hash : • key : user id • ﬁelds : friends ids • value : add time

=feed+ +• redis• hash : • hset fromuid.following touid addtime • hset touid.follower fromuid addtime • hgetAll fromuid. following • hgetAll touid.follower ? • 11,704,598 @Wed Sep 7 21:46:33 CST 2011

=feed+ +• hash: get one or get all• hash-max-zip-size • 64 -> 256 1/3 • cpu• hgetAll cost too much cpu • add mc• high delay

=feed+ +

=feed+ +• • cache ? waste too much mem • storage ? • rdb may lost data • aof r/w too slow, recover too slow • all data in mem, waste money • HA : master slave ? NO WAY • memory fragment

=feed+ +

=feed+ +• • mysql binlog >> queue >> Java Processor >> redis • mysql binlog >> trigger >> redis

=feed+ +• • redis@weibo for now: • TB • growing fast

=feed+ +• •@ mysql modiﬁed + innodb • following list of a user in one column • still under dev

=feed+ +•

=feed+ +• • • •@ • •

=feed+ +

=feed+ +• • • @ • • • js

=feed+ +• mc + queue + mysql• mysql• mc mysql• mc

=feed+ +• • • • •

=feed+ +• redis• • k-v , 100 byte per k-v • mc • • hash , hget pipeline slow

=feed+ +• • rdb ? may lost data • aof ? grow too fast (4G/day) • bgsave/bgrewriteaof inﬂuence parent

=feed+ +

=feed+ +• redis rolling • • • • key

=feed+ +• • rediscounter @ • array , not linked list • malloc all mem when start • hash key to position • write disk: asyn & slow down • add position to aof ﬁle

=feed+ +• • rediscounter + innodb • auto roll cold data to disk

=feed+ +• • @TimYang • @XiaoJunHong • • redis (delay) - mc (throughput) - java hash map

• • cache • storage

• • • • + •

• • • / • •

• • • rdb or aof • •

• • • rehash

• HA / Cluster • Redis • Jedis • • •

• CPU • Redis • hset with big hash-max-zip-size • hgetAll • mc

• • • • •

Thanks PS. We are hiring !contact me via @

Q &A @http://weibo.com/tangﬂ http://fulin.org

http://blog.nosqlfan.com	9587
http://www.mysqlops.com	2505
http://www.bsdmap.com	845
http://simple-is-better.com	571
http://huangyong.tk	389
http://blog.haohtml.com	347
http://www.dcshi.com	289
http://www.tblog.org	262
http://www.oschina.net	232
http://192.168.1.101	182
http://hadoopguru.info	154
http://localhost	137
http://www.51testing.com	127
http://yunti.sinaapp.com	85
http://www.kuqin.com	62
http://feed.feedsky.com	56
http://cache.baidu.com	44
http://sandbox.clipboard.com	41
http://blog.bypat.com	33
http://58.215.188.40	31
http://xianguo.com	30
http://www.xwuxin.com	24
http://zhaicl.com	21
http://gofier.com	17
http://m.oschina.net	17
http://itindex.net	15
http://itwwt.com	14
http://reader.youdao.com	12
http://highscalability.info	11
http://www.simple-is-better.com	10
http://blog.chedushi.com	10
http://zhuaxia.com	9
http://wew.bsdmap.com	7
http://dandan.nonobo.com	7
http://www.gofier.com	7
http://www.php163.com	6
http://stage.clpbrd.com	6
http://web.mail.bsdmap.com	5
http://1qibu.com	5
http://207.46.192.232	5
http://www.zhuaxia.com	4
http://www.mysqlops.com HTTP	4
http://www2.bsdmap.com	4
http://webcache.googleusercontent.com	4
http://w3w.bsdmap.com	3
http://qww.bsdmap.com	3
http://http.bsdmap.com	3
http://w3ww.bsdmap.com	3
http://222.bsdmap.com	3
http://www.1qibu.com	3
http://wap.oschina.net	2

新浪微博开放平台Redis实战

by mysqlops

Accessibility

Categories

Upload Details

Usage Rights

72 Embeds 16,276

Statistics